2024-05-21T17:31:25.675 INFO:root:teuthology version: 0.0.1.dev287+gb9e3da8 2024-05-21T17:31:25.675 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/mgr/{clusters/{2-node-mgr} debug/mgr distro/{centos_latest} mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/{1-install 2-ceph 3-mgrmodules 4-units/workunits}} --owner scheduled_yuriw@teuthology --name yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528 --verbose -- /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528/orig.config.yaml 2024-05-21T17:31:25.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T17:31:25.791 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528 branch: wip-yuri7-testing-2024-05-20-1227 description: rados/mgr/{clusters/{2-node-mgr} debug/mgr distro/{centos_latest} mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/{1-install 2-ceph 3-mgrmodules 4-units/workunits}} email: yweinste@redhat.com first_in_suite: false job_id: '7717528' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 2 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri7-testing-2024-05-20-1227 ceph: conf: client: debug client: 20 debug mgrc: 20 debug ms: 1 global: mon election default strategy: 1 mds: debug mgrc: 20 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mgr ttl cache expire seconds: 0 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug mgrc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri7-testing-2024-05-20-1227 sha1: 9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mgr.x - mon.a - mon.c - mds.a - mds.c - osd.0 - client.0 - - mgr.y - mgr.z - mon.b - mds.b - osd.1 - osd.2 - osd.3 - client.1 seed: 4736 sha1: 9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri7-testing-2024-05-20-1227 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVpevLpr+7TbmJfvsgGCXP8LXqBJdnA+8DctMAGvYIiAvkNib/r6aPZkRp1QMdGmeAzBdE4B5EcqTsHLd04hpM= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwQaEx4f5aPRqCDCqzJi44+In4ryYKNFY4kyLj+LNZ/Tlhk3ZPZrHqWcIJYL7k+y/YJ06lUJYpAYXBP1LHNPMA= tasks: - install: extra_system_packages: deb: - sqlite3 rpm: - sqlite-devel - ceph: log-ignorelist: - overall HEALTH_ - \(MGR_DOWN\) - \(PG_ - replacing it with standby - No standby daemons available - \(POOL_APP_NOT_ENABLED\) wait-for-scrub: false - sequential: - mgrmodules - workunit: clients: client.0: - mgr/test_localpool.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a timestamp: 2024-05-21_01:07:55 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-21T17:31:25.791 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/qa; will attempt to use it 2024-05-21T17:31:25.792 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/qa/tasks 2024-05-21T17:31:25.792 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-21T17:31:25.793 INFO:teuthology.task.internal:Checking packages... 2024-05-21T17:31:25.814 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7' 2024-05-21T17:31:25.815 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-21T17:31:25.815 INFO:teuthology.packaging:ref: None 2024-05-21T17:31:25.815 INFO:teuthology.packaging:tag: None 2024-05-21T17:31:25.815 INFO:teuthology.packaging:branch: wip-yuri7-testing-2024-05-20-1227 2024-05-21T17:31:25.815 INFO:teuthology.packaging:sha1: 9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 2024-05-21T17:31:25.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri7-testing-2024-05-20-1227 2024-05-21T17:31:26.032 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3870.g9ef9bcfd 2024-05-21T17:31:26.034 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-21T17:31:26.041 INFO:teuthology.task.internal:no buildpackages task found 2024-05-21T17:31:26.041 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-21T17:31:26.084 INFO:teuthology.task.internal:Saving configuration 2024-05-21T17:31:26.096 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-21T17:31:26.103 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-21T17:31:26.128 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 17:26:17.118051', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVpevLpr+7TbmJfvsgGCXP8LXqBJdnA+8DctMAGvYIiAvkNib/r6aPZkRp1QMdGmeAzBdE4B5EcqTsHLd04hpM='} 2024-05-21T17:31:26.153 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-21 17:26:17.115035', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwQaEx4f5aPRqCDCqzJi44+In4ryYKNFY4kyLj+LNZ/Tlhk3ZPZrHqWcIJYL7k+y/YJ06lUJYpAYXBP1LHNPMA='} 2024-05-21T17:31:26.153 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-21T17:31:26.159 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'] 2024-05-21T17:31:26.159 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1'] 2024-05-21T17:31:26.159 INFO:teuthology.run_tasks:Running task console_log... 2024-05-21T17:31:26.219 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f992a8e7be0>, signals=[15]) 2024-05-21T17:31:26.219 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-21T17:31:26.225 INFO:teuthology.task.internal:Opening connections... 2024-05-21T17:31:26.225 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-05-21T17:31:26.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:31:26.301 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-05-21T17:31:26.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:31:26.378 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-21T17:31:26.384 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-05-21T17:31:26.403 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-05-21T17:31:26.404 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-05-21T17:31:26.461 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T17:31:26.462 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T17:31:26.463 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T17:31:26.463 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-05-21T17:31:26.478 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-05-21T17:31:26.495 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-05-21T17:31:26.496 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-05-21T17:31:26.552 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-05-21T17:31:26.553 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-21T17:31:26.554 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-21T17:31:26.554 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-21T17:31:26.554 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-05-21T17:31:26.587 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-21T17:31:26.595 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-21T17:31:26.632 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-21T17:31:26.633 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-05-21T17:31:26.635 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-05-21T17:31:26.651 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-21T17:31:26.657 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-21T17:31:26.658 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-05-21T17:31:26.693 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-05-21T17:31:26.724 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-21T17:31:26.778 INFO:teuthology.run_tasks:Running task kernel... 2024-05-21T17:31:26.793 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-21T17:31:26.793 INFO:teuthology.task.kernel:config {'mgr.x': {'kdb': 1, 'sha1': 'distro'}, 'mgr.y': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-21T17:31:26.794 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T17:31:26.794 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-21T17:31:26.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T17:31:26.811 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-05-21T17:31:26.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T17:31:26.812 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-05-21T17:31:26.868 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-437.el9.x86_64 2024-05-21T17:31:26.869 INFO:teuthology.task.kernel:Running kernel on smithi203: 5.14.0-437.el9.x86_64 2024-05-21T17:31:26.869 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum install -y kernel 2024-05-21T17:31:26.870 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-437.el9.x86_64 2024-05-21T17:31:26.870 INFO:teuthology.task.kernel:Running kernel on smithi003: 5.14.0-437.el9.x86_64 2024-05-21T17:31:26.870 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y kernel 2024-05-21T17:31:28.962 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-21T17:31:29.022 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-21T17:31:30.790 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-05-21T17:31:33.672 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 19 MB 00:03 2024-05-21T17:31:35.165 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-05-21T17:31:36.616 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-05-21T17:31:38.012 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-21T17:31:38.913 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-05-21T17:31:41.856 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 16 kB 00:01 2024-05-21T17:31:43.326 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-21T17:31:45.182 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 21 MB 00:03 2024-05-21T17:31:45.290 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-21T17:31:45.291 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-21T17:31:45.387 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-05-21T17:31:45.401 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-05-21T17:31:45.401 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-05-21T17:31:45.401 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-05-21T17:31:45.401 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-05-21T17:31:45.401 INFO:teuthology.orchestra.run.smithi003.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-21T17:31:45.402 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-05-21T17:31:45.402 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-21T17:31:45.402 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-21T17:31:45.402 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-21T17:31:45.402 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:31:45.402 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-05-21T17:31:45.402 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-05-21T17:31:45.402 INFO:teuthology.orchestra.run.smithi003.stdout:Install 4 Packages 2024-05-21T17:31:45.402 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:31:45.403 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 78 M 2024-05-21T17:31:45.403 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 123 M 2024-05-21T17:31:45.403 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-05-21T17:31:46.477 INFO:teuthology.orchestra.run.smithi003.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 553 kB/s | 576 kB 00:01 2024-05-21T17:31:47.594 INFO:teuthology.orchestra.run.smithi003.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 7.2 MB/s | 16 MB 00:02 2024-05-21T17:31:48.227 INFO:teuthology.orchestra.run.smithi003.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 12 MB/s | 34 MB 00:02 2024-05-21T17:31:49.236 INFO:teuthology.orchestra.run.smithi003.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 10 MB/s | 28 MB 00:02 2024-05-21T17:31:49.236 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-05-21T17:31:49.236 INFO:teuthology.orchestra.run.smithi003.stdout:Total 20 MB/s | 78 MB 00:03 2024-05-21T17:31:49.945 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-05-21T17:31:50.375 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-05-21T17:31:50.375 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-05-21T17:31:50.457 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-21T17:31:51.320 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-05-21T17:31:51.320 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-05-21T17:31:52.288 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-21T17:31:52.288 INFO:teuthology.orchestra.run.smithi203.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-21T17:31:52.385 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-21T17:31:52.395 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-21T17:31:52.395 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2024-05-21T17:31:52.395 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-21T17:31:52.395 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-21T17:31:52.395 INFO:teuthology.orchestra.run.smithi203.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-21T17:31:52.395 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-21T17:31:52.395 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-21T17:31:52.396 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-21T17:31:52.396 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-21T17:31:52.396 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:31:52.396 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-21T17:31:52.396 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2024-05-21T17:31:52.396 INFO:teuthology.orchestra.run.smithi203.stdout:Install 4 Packages 2024-05-21T17:31:52.396 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:31:52.396 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 78 M 2024-05-21T17:31:52.397 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 123 M 2024-05-21T17:31:52.397 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-21T17:31:52.609 INFO:teuthology.orchestra.run.smithi203.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 3.2 MB/s | 576 kB 00:00 2024-05-21T17:31:53.026 INFO:teuthology.orchestra.run.smithi203.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-05-21T17:31:53.609 INFO:teuthology.orchestra.run.smithi203.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 29 MB/s | 34 MB 00:01 2024-05-21T17:31:53.862 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-05-21T17:31:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 22 MB/s | 28 MB 00:01 2024-05-21T17:31:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-21T17:31:53.876 INFO:teuthology.orchestra.run.smithi203.stdout:Total 53 MB/s | 78 MB 00:01 2024-05-21T17:31:54.529 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-21T17:31:54.561 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-21T17:31:54.619 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T17:31:54.906 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-21T17:31:54.906 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-21T17:31:55.834 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-21T17:31:55.834 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-21T17:31:57.396 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T17:31:57.522 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T17:31:58.241 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-21T17:31:58.867 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-21T17:31:58.945 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T17:32:01.604 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T17:32:01.749 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T17:32:02.591 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T17:32:02.662 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:06.454 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T17:32:06.561 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:07.821 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:11.273 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:38.861 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:38.902 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:39.290 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:39.291 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-21T17:32:39.291 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T17:32:39.291 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T17:32:40.100 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:40.101 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:32:40.101 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-05-21T17:32:40.101 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-21T17:32:40.101 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-21T17:32:40.101 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-21T17:32:40.101 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-21T17:32:40.101 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:32:40.101 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-05-21T17:32:40.433 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T17:32:40.544 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T17:32:40.545 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T17:32:40.545 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-21T17:32:40.545 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-21T17:32:40.545 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-21T17:32:40.545 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-05-21T17:32:40.545 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2024-05-21T17:32:40.546 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T17:32:40.597 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T17:32:40.598 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T17:32:40.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2024-05-21T17:32:40.629 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2024-05-21T17:32:40.629 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2024-05-21T17:32:40.629 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.06 2024-05-21T17:32:40.629 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 68.el9 2024-05-21T17:32:40.629 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2024-05-21T17:32:40.629 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-21T17:32:40.629 INFO:teuthology.orchestra.run.smithi003.stdout:Group : Unspecified 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 8264254 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : builder@centos.org 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:32:40.630 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-21T17:32:40.631 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-21T17:32:40.631 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-21T17:32:40.631 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2024-05-21T17:32:40.631 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:32:40.631 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2024-05-21T17:32:40.632 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-21T17:32:40.632 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 5.14.0-447.el9.x86_64 2024-05-21T17:32:40.632 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-21T17:32:41.210 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-05-21T17:32:41.257 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:41.542 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:41.543 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-21T17:32:41.543 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-21T17:32:41.543 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-21T17:32:42.303 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-21T17:32:42.303 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:32:42.303 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-21T17:32:42.303 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-21T17:32:42.303 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-21T17:32:42.304 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-21T17:32:42.304 INFO:teuthology.orchestra.run.smithi203.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-21T17:32:42.304 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:32:42.304 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-21T17:32:42.661 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T17:32:42.713 INFO:teuthology.orchestra.run.smithi003.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-21T17:32:42.732 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T17:32:42.732 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T17:32:42.732 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-21T17:32:42.732 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-21T17:32:42.732 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-21T17:32:42.732 INFO:teuthology.task.kernel:Installing distro kernel on mgr.y... 2024-05-21T17:32:42.732 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi203.front.sepia.ceph.com, path=None, version=distro) 2024-05-21T17:32:42.733 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q kernel | sort -rV | head -n 1 2024-05-21T17:32:42.757 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-05-21T17:32:42.764 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-05-21T17:32:42.781 INFO:teuthology.orchestra.run.smithi203.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-21T17:32:42.782 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-21T17:32:42.782 DEBUG:teuthology.orchestra.run.smithi203:> sudo rpm -qi grub2-tools 2024-05-21T17:32:42.787 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.bBq7LYR9Xd 2024-05-21T17:32:42.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bBq7LYR9Xd 2024-05-21T17:32:42.822 INFO:teuthology.orchestra.run.smithi203.stdout:Name : grub2-tools 2024-05-21T17:32:42.822 INFO:teuthology.orchestra.run.smithi203.stdout:Epoch : 1 2024-05-21T17:32:42.822 INFO:teuthology.orchestra.run.smithi203.stdout:Version : 2.06 2024-05-21T17:32:42.822 INFO:teuthology.orchestra.run.smithi203.stdout:Release : 68.el9 2024-05-21T17:32:42.822 INFO:teuthology.orchestra.run.smithi203.stdout:Architecture: x86_64 2024-05-21T17:32:42.822 INFO:teuthology.orchestra.run.smithi203.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-21T17:32:42.822 INFO:teuthology.orchestra.run.smithi203.stdout:Group : Unspecified 2024-05-21T17:32:42.822 INFO:teuthology.orchestra.run.smithi203.stdout:Size : 8264254 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:License : GPLv3+ 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:Packager : builder@centos.org 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:Vendor : CentOS 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:Summary : Support tools for GRUB. 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:Description : 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-21T17:32:42.823 INFO:teuthology.orchestra.run.smithi203.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-21T17:32:42.824 INFO:teuthology.orchestra.run.smithi203.stdout:hardware devices. 2024-05-21T17:32:42.824 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:32:42.824 INFO:teuthology.orchestra.run.smithi203.stdout:This subpackage provides tools for support of all platforms. 2024-05-21T17:32:42.824 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-21T17:32:42.825 INFO:teuthology.task.kernel:Updating grub on smithi203 to boot 5.14.0-447.el9.x86_64 2024-05-21T17:32:42.825 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-21T17:32:42.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.bBq7LYR9Xd 2024-05-21T17:32:42.991 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.bBq7LYR9Xd is 6KB 2024-05-21T17:32:43.011 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.bBq7LYR9Xd 2024-05-21T17:32:43.026 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2024-05-21T17:32:43.105 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-21T17:32:43.105 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-21T17:32:43.105 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-21T17:32:43.105 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-21T17:32:43.106 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-21T17:32:43.245 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-05-21T17:32:43.469 INFO:teuthology.orchestra.run.smithi203.stderr:Generating grub configuration file ... 2024-05-21T17:32:44.802 INFO:teuthology.orchestra.run.smithi203.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-21T17:32:44.844 INFO:teuthology.orchestra.run.smithi203.stderr:done 2024-05-21T17:32:44.846 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-05-21T17:32:44.861 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.sFyjBu7LKy 2024-05-21T17:32:44.862 DEBUG:teuthology.orchestra.run.smithi203:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.sFyjBu7LKy 2024-05-21T17:32:44.926 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.sFyjBu7LKy 2024-05-21T17:32:45.061 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.sFyjBu7LKy is 6KB 2024-05-21T17:32:45.110 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.sFyjBu7LKy 2024-05-21T17:32:45.125 DEBUG:teuthology.orchestra.run.smithi203:> sudo /bin/ls /boot/loader/entries || true 2024-05-21T17:32:45.186 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-21T17:32:45.187 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-21T17:32:45.187 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-21T17:32:45.187 INFO:teuthology.orchestra.run.smithi203.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-21T17:32:45.188 DEBUG:teuthology.orchestra.run.smithi203:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-21T17:32:45.327 DEBUG:teuthology.orchestra.run.smithi203:> sudo shutdown -r now 2024-05-21T17:33:13.274 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-05-21T17:33:13.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-21T17:33:13.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:33:15.330 INFO:teuthology.task.kernel:Checking client mgr.y for new kernel version... 2024-05-21T17:33:15.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-21T17:33:15.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:33:31.811 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-05-21T17:33:33.854 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-05-21T17:33:40.817 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-21T17:33:40.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:33:42.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-21T17:33:42.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:33:45.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-05-21T17:33:57.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-21T17:33:57.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:34:16.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-05-21T17:34:31.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-21T17:34:31.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:34:34.968 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-05-21T17:34:35.418 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-21T17:34:35.419 INFO:teuthology.task.kernel:Checking kernel version of mgr.y, want "5.14.0-447.el9.x86_64"... 2024-05-21T17:34:35.419 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-05-21T17:34:35.433 INFO:teuthology.orchestra.run.smithi203.stdout:5.14.0-447.el9.x86_64 2024-05-21T17:34:35.434 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-21T17:34:35.434 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-21T17:34:35.434 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-21T17:34:36.434 INFO:teuthology.task.kernel:Enabling kdb on mgr.y... 2024-05-21T17:34:36.435 DEBUG:teuthology.orchestra.run.smithi203:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-21T17:34:36.504 INFO:teuthology.orchestra.run.smithi203.stdout:ttyS1 2024-05-21T17:34:36.546 DEBUG:teuthology.parallel:result is None 2024-05-21T17:34:40.820 DEBUG:teuthology.orchestra.remote:timed out 2024-05-21T17:34:52.820 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-21T17:34:52.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:34:53.122 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-05-21T17:34:53.490 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-21T17:34:53.491 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-447.el9.x86_64"... 2024-05-21T17:34:53.491 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-05-21T17:34:53.507 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-447.el9.x86_64 2024-05-21T17:34:53.508 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-21T17:34:53.508 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-21T17:34:53.508 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-21T17:34:54.509 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-05-21T17:34:54.509 DEBUG:teuthology.orchestra.run.smithi003:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-21T17:34:54.582 INFO:teuthology.orchestra.run.smithi003.stdout:ttyS1 2024-05-21T17:34:54.609 DEBUG:teuthology.parallel:result is None 2024-05-21T17:34:54.610 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-21T17:34:54.617 INFO:teuthology.task.internal:Creating test directory... 2024-05-21T17:34:54.617 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T17:34:54.619 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-21T17:34:54.640 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-21T17:34:54.646 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-21T17:34:54.652 INFO:teuthology.task.internal:Creating archive directory... 2024-05-21T17:34:54.652 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T17:34:54.678 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-21T17:34:54.714 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-21T17:34:54.720 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-21T17:34:54.720 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T17:34:54.750 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-21T17:34:54.781 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T17:34:54.782 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T17:34:54.793 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T17:34:54.793 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-21T17:34:54.795 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-21T17:34:54.802 INFO:teuthology.task.internal:Configuring sudo... 2024-05-21T17:34:54.802 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T17:34:54.838 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-21T17:34:54.866 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-21T17:34:54.875 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-21T17:34:54.875 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T17:34:54.907 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-21T17:34:54.924 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T17:34:55.005 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T17:34:55.056 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T17:34:55.141 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T17:34:55.182 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:34:55.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T17:34:55.249 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T17:34:55.295 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-21T17:34:55.357 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T17:34:55.401 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-21T17:34:55.446 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:34:55.446 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-21T17:34:55.513 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-05-21T17:34:55.515 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-05-21T17:34:55.553 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T17:34:55.607 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T17:34:55.899 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-21T17:34:55.906 INFO:teuthology.task.internal:Starting timer... 2024-05-21T17:34:55.907 INFO:teuthology.run_tasks:Running task pcp... 2024-05-21T17:34:55.916 INFO:teuthology.run_tasks:Running task selinux... 2024-05-21T17:34:55.924 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-21T17:34:55.924 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2024-05-21T17:34:55.994 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2024-05-21T17:34:55.996 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-05-21T17:34:56.049 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-05-21T17:34:56.051 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-21T17:34:56.051 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2024-05-21T17:34:56.096 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2024-05-21T17:34:56.096 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-05-21T17:34:56.129 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-05-21T17:34:56.130 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-05-21T17:34:56.130 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T17:34:56.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T17:34:56.164 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T17:34:56.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T17:34:56.197 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-21T17:34:56.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2024-05-21T17:34:56.233 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-05-21T17:34:56.263 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-21T17:34:56.273 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-21T17:34:56.285 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-21T17:34:56.286 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-21T17:39:17.036 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-05-21T17:39:17.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-21T17:39:17.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:39:17.115 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-05-21T17:39:17.198 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-21T17:39:17.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-21T17:39:17.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-21T17:39:17.279 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-05-21T17:39:17.357 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-05-21T17:39:17.357 INFO:teuthology.run_tasks:Running task clock... 2024-05-21T17:39:17.368 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-21T17:39:17.369 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T17:39:17.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T17:39:17.371 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-21T17:39:17.372 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T17:39:17.402 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T17:39:17.418 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T17:39:17.427 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-21T17:39:17.442 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-21T17:39:17.444 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2024-05-21T17:39:17.455 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2024-05-21T17:39:17.466 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-05-21T17:39:17.471 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T17:39:17.477 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-05-21T17:39:17.485 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T17:39:17.490 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-21T17:39:17.504 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-21T17:39:17.544 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-05-21T17:39:17.547 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T17:39:17.548 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-05-21T17:39:17.548 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T17:39:17.548 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T17:39:17.548 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T17:39:17.548 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T17:39:17.561 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-05-21T17:39:17.564 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T17:39:17.564 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-05-21T17:39:17.564 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T17:39:17.565 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T17:39:17.565 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T17:39:17.565 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T17:39:17.566 INFO:teuthology.run_tasks:Running task install... 2024-05-21T17:39:17.573 DEBUG:teuthology.task.install:project ceph 2024-05-21T17:39:17.574 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7'}} 2024-05-21T17:39:17.574 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7'} 2024-05-21T17:39:17.574 INFO:teuthology.task.install:Using flavor: default 2024-05-21T17:39:17.579 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-21T17:39:17.579 INFO:teuthology.task.install:extra packages: [] 2024-05-21T17:39:17.580 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7', 'tag': None, 'wait_for_package': False} 2024-05-21T17:39:17.581 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 2024-05-21T17:39:17.582 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7', 'tag': None, 'wait_for_package': False} 2024-05-21T17:39:17.583 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 2024-05-21T17:39:17.726 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-05-20-1227/9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/centos/9/flavors/default/ 2024-05-21T17:39:17.726 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3870.g9ef9bcfd 2024-05-21T17:39:17.870 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-05-20-1227/9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/centos/9/flavors/default/ 2024-05-21T17:39:17.870 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3870.g9ef9bcfd 2024-05-21T17:39:18.004 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-05-20-1227/9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-05-20-1227/9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-05-20-1227/9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T17:39:18.004 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:39:18.004 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T17:39:18.033 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, sqlite-devel, sqlite-devel, sqlite-devel on remote rpm x86_64 2024-05-21T17:39:18.033 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T17:39:18.098 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T17:39:18.101 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-05-20-1227/9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-05-20-1227/9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2024-05-20-1227/9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-21T17:39:18.101 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:39:18.101 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-21T17:39:18.129 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, sqlite-devel, sqlite-devel, sqlite-devel, sqlite-devel on remote rpm x86_64 2024-05-21T17:39:18.129 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-21T17:39:18.173 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T17:39:18.199 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-21T17:39:18.238 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-05-21T17:39:18.239 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-05-21T17:39:18.278 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-21T17:39:18.344 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2024-05-21T17:39:18.346 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2024-05-21T17:39:18.463 INFO:teuthology.orchestra.run.smithi203.stdout:45 files removed 2024-05-21T17:39:18.490 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel 2024-05-21T17:39:18.583 INFO:teuthology.orchestra.run.smithi003.stdout:45 files removed 2024-05-21T17:39:18.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel 2024-05-21T17:39:19.260 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 169 kB/s | 84 kB 00:00 2024-05-21T17:39:19.384 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 173 kB/s | 84 kB 00:00 2024-05-21T17:39:19.653 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-21T17:39:19.767 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-21T17:39:19.987 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-05-21T17:39:20.134 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-05-21T17:39:20.551 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 22 MB/s | 8.1 MB 00:00 2024-05-21T17:39:22.895 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.1 MB 00:02 2024-05-21T17:39:23.044 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-05-21T17:39:25.005 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-05-21T17:39:27.703 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.2 MB 00:00 2024-05-21T17:39:29.196 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-05-21T17:39:29.772 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.2 MB 00:01 2024-05-21T17:39:31.175 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-21T17:39:31.358 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 16 kB 00:00 2024-05-21T17:39:36.531 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-21T17:39:38.834 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T17:39:38.834 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T17:39:38.898 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-05-21T17:39:38.914 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-05-21T17:39:38.914 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-05-21T17:39:38.914 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-05-21T17:39:38.914 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-05-21T17:39:38.914 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 6.6 k 2024-05-21T17:39:38.914 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 5.4 M 2024-05-21T17:39:38.914 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 890 k 2024-05-21T17:39:38.914 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 145 k 2024-05-21T17:39:38.915 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 1.5 M 2024-05-21T17:39:38.915 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 143 k 2024-05-21T17:39:38.915 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 3.5 M 2024-05-21T17:39:38.915 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 7.4 M 2024-05-21T17:39:38.915 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 49 k 2024-05-21T17:39:38.915 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 11 M 2024-05-21T17:39:38.915 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 49 M 2024-05-21T17:39:38.915 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 273 k 2024-05-21T17:39:38.915 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 764 k 2024-05-21T17:39:38.915 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 32 k 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 747 k 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 127 k 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 162 k 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 321 k 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 300 k 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 99 k 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 85 k 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 3.1 M 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 170 k 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout: sqlite-devel x86_64 3.34.1-7.el9 appstream 134 k 2024-05-21T17:39:38.916 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-05-21T17:39:38.917 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 3.4 M 2024-05-21T17:39:38.917 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 3.1 M 2024-05-21T17:39:38.917 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-05-21T17:39:38.917 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T17:39:38.917 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T17:39:38.917 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 21 M 2024-05-21T17:39:38.917 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 34 k 2024-05-21T17:39:38.917 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 2.4 M 2024-05-21T17:39:38.917 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 274 k 2024-05-21T17:39:38.917 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 4.7 M 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 17 M 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 16 k 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 25 k 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T17:39:38.918 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 163 k 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 501 k 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 5.4 M 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T17:39:38.919 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T17:39:38.920 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 45 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 136 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T17:39:38.921 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T17:39:38.922 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T17:39:38.923 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T17:39:38.924 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout:Installing weak dependencies: 2024-05-21T17:39:38.925 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T17:39:38.926 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:39:38.926 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-05-21T17:39:38.926 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-05-21T17:39:38.926 INFO:teuthology.orchestra.run.smithi003.stdout:Install 126 Packages 2024-05-21T17:39:38.926 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 2 Packages 2024-05-21T17:39:38.926 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:39:38.926 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 203 M 2024-05-21T17:39:38.927 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-05-21T17:39:39.532 INFO:teuthology.orchestra.run.smithi003.stdout:(1/128): ceph-19.0.0-3870.g9ef9bcfd.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-05-21T17:39:39.535 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 2.6 MB/s | 21 MB 00:08 2024-05-21T17:39:39.766 INFO:teuthology.orchestra.run.smithi003.stdout:(2/128): ceph-fuse-19.0.0-3870.g9ef9bcfd.el9.x8 3.7 MB/s | 890 kB 00:00 2024-05-21T17:39:39.832 INFO:teuthology.orchestra.run.smithi003.stdout:(3/128): ceph-immutable-object-cache-19.0.0-387 2.1 MB/s | 145 kB 00:00 2024-05-21T17:39:40.125 INFO:teuthology.orchestra.run.smithi003.stdout:(4/128): ceph-mds-19.0.0-3870.g9ef9bcfd.el9.x86 8.1 MB/s | 2.4 MB 00:00 2024-05-21T17:39:40.366 INFO:teuthology.orchestra.run.smithi003.stdout:(5/128): ceph-mgr-19.0.0-3870.g9ef9bcfd.el9.x86 6.3 MB/s | 1.5 MB 00:00 2024-05-21T17:39:40.926 INFO:teuthology.orchestra.run.smithi003.stdout:(6/128): ceph-mon-19.0.0-3870.g9ef9bcfd.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-05-21T17:39:41.000 INFO:teuthology.orchestra.run.smithi003.stdout:(7/128): ceph-base-19.0.0-3870.g9ef9bcfd.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-05-21T17:39:41.234 INFO:teuthology.orchestra.run.smithi003.stdout:(8/128): ceph-common-19.0.0-3870.g9ef9bcfd.el9. 11 MB/s | 21 MB 00:01 2024-05-21T17:39:41.292 INFO:teuthology.orchestra.run.smithi003.stdout:(9/128): ceph-selinux-19.0.0-3870.g9ef9bcfd.el9 433 kB/s | 25 kB 00:00 2024-05-21T17:39:42.401 INFO:teuthology.orchestra.run.smithi003.stdout:(10/128): ceph-radosgw-19.0.0-3870.g9ef9bcfd.el 7.6 MB/s | 11 MB 00:01 2024-05-21T17:39:42.576 INFO:teuthology.orchestra.run.smithi003.stdout:(11/128): ceph-osd-19.0.0-3870.g9ef9bcfd.el9.x8 10 MB/s | 17 MB 00:01 2024-05-21T17:39:42.601 INFO:teuthology.orchestra.run.smithi003.stdout:(12/128): libcephfs-devel-19.0.0-3870.g9ef9bcfd 158 kB/s | 32 kB 00:00 2024-05-21T17:39:42.685 INFO:teuthology.orchestra.run.smithi003.stdout:(13/128): libcephfs2-19.0.0-3870.g9ef9bcfd.el9. 6.7 MB/s | 747 kB 00:00 2024-05-21T17:39:42.710 INFO:teuthology.orchestra.run.smithi003.stdout:(14/128): libcephsqlite-19.0.0-3870.g9ef9bcfd.e 1.5 MB/s | 163 kB 00:00 2024-05-21T17:39:42.744 INFO:teuthology.orchestra.run.smithi003.stdout:(15/128): librados-devel-19.0.0-3870.g9ef9bcfd. 2.1 MB/s | 127 kB 00:00 2024-05-21T17:39:42.803 INFO:teuthology.orchestra.run.smithi003.stdout:(16/128): libradosstriper1-19.0.0-3870.g9ef9bcf 5.3 MB/s | 501 kB 00:00 2024-05-21T17:39:42.886 INFO:teuthology.orchestra.run.smithi003.stdout:(17/128): python3-ceph-argparse-19.0.0-3870.g9e 542 kB/s | 45 kB 00:00 2024-05-21T17:39:42.970 INFO:teuthology.orchestra.run.smithi003.stdout:(18/128): python3-ceph-common-19.0.0-3870.g9ef9 1.6 MB/s | 136 kB 00:00 2024-05-21T17:39:43.054 INFO:teuthology.orchestra.run.smithi003.stdout:(19/128): python3-cephfs-19.0.0-3870.g9ef9bcfd. 1.9 MB/s | 162 kB 00:00 2024-05-21T17:39:43.154 INFO:teuthology.orchestra.run.smithi003.stdout:(20/128): python3-rados-19.0.0-3870.g9ef9bcfd.e 3.1 MB/s | 321 kB 00:00 2024-05-21T17:39:43.271 INFO:teuthology.orchestra.run.smithi003.stdout:(21/128): librgw2-19.0.0-3870.g9ef9bcfd.el9.x86 10 MB/s | 5.4 MB 00:00 2024-05-21T17:39:43.296 INFO:teuthology.orchestra.run.smithi003.stdout:(22/128): python3-rbd-19.0.0-3870.g9ef9bcfd.el9 2.1 MB/s | 300 kB 00:00 2024-05-21T17:39:43.321 INFO:teuthology.orchestra.run.smithi003.stdout:(23/128): python3-rgw-19.0.0-3870.g9ef9bcfd.el9 1.9 MB/s | 99 kB 00:00 2024-05-21T17:39:43.380 INFO:teuthology.orchestra.run.smithi003.stdout:(24/128): rbd-fuse-19.0.0-3870.g9ef9bcfd.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-21T17:39:43.472 INFO:teuthology.orchestra.run.smithi003.stdout:(25/128): rbd-nbd-19.0.0-3870.g9ef9bcfd.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-21T17:39:43.572 INFO:teuthology.orchestra.run.smithi003.stdout:(26/128): ceph-grafana-dashboards-19.0.0-3870.g 343 kB/s | 34 kB 00:00 2024-05-21T17:39:43.656 INFO:teuthology.orchestra.run.smithi003.stdout:(27/128): rbd-mirror-19.0.0-3870.g9ef9bcfd.el9. 9.2 MB/s | 3.1 MB 00:00 2024-05-21T17:39:43.681 INFO:teuthology.orchestra.run.smithi003.stdout:(28/128): ceph-mgr-cephadm-19.0.0-3870.g9ef9bcf 1.3 MB/s | 143 kB 00:00 2024-05-21T17:39:44.207 INFO:teuthology.orchestra.run.smithi003.stdout:(29/128): ceph-mgr-dashboard-19.0.0-3870.g9ef9b 6.4 MB/s | 3.5 MB 00:00 2024-05-21T17:39:44.290 INFO:teuthology.orchestra.run.smithi003.stdout:(30/128): ceph-mgr-modules-core-19.0.0-3870.g9e 3.2 MB/s | 274 kB 00:00 2024-05-21T17:39:44.357 INFO:teuthology.orchestra.run.smithi003.stdout:(31/128): ceph-mgr-rook-19.0.0-3870.g9ef9bcfd.e 741 kB/s | 49 kB 00:00 2024-05-21T17:39:44.458 INFO:teuthology.orchestra.run.smithi003.stdout:(32/128): ceph-mgr-diskprediction-local-19.0.0- 9.6 MB/s | 7.4 MB 00:00 2024-05-21T17:39:44.474 INFO:teuthology.orchestra.run.smithi003.stdout:(33/128): ceph-prometheus-alerts-19.0.0-3870.g9 140 kB/s | 16 kB 00:00 2024-05-21T17:39:44.533 INFO:teuthology.orchestra.run.smithi003.stdout:(34/128): ceph-volume-19.0.0-3870.g9ef9bcfd.el9 3.6 MB/s | 273 kB 00:00 2024-05-21T17:39:44.600 INFO:teuthology.orchestra.run.smithi003.stdout:(35/128): cephadm-19.0.0-3870.g9ef9bcfd.el9.noa 6.0 MB/s | 764 kB 00:00 2024-05-21T17:39:44.683 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-21T17:39:44.950 INFO:teuthology.orchestra.run.smithi003.stdout:(36/128): fuse-2.9.9-15.el9.x86_64.rpm 193 kB/s | 80 kB 00:00 2024-05-21T17:39:45.134 INFO:teuthology.orchestra.run.smithi003.stdout:(37/128): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-21T17:39:45.326 INFO:teuthology.orchestra.run.smithi003.stdout:(38/128): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-05-21T17:39:45.810 INFO:teuthology.orchestra.run.smithi003.stdout:(39/128): ceph-test-19.0.0-3870.g9ef9bcfd.el9.x 11 MB/s | 49 MB 00:04 2024-05-21T17:39:45.843 INFO:teuthology.orchestra.run.smithi003.stdout:(40/128): libgfortran-11.4.1-3.el9.x86_64.rpm 646 kB/s | 803 kB 00:01 2024-05-21T17:39:45.960 INFO:teuthology.orchestra.run.smithi003.stdout:(41/128): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-21T17:39:46.010 INFO:teuthology.orchestra.run.smithi003.stdout:(42/128): python3-cryptography-36.0.1-4.el9.x86 1.8 MB/s | 1.2 MB 00:00 2024-05-21T17:39:46.069 INFO:teuthology.orchestra.run.smithi003.stdout:(43/128): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-21T17:39:46.119 INFO:teuthology.orchestra.run.smithi003.stdout:(44/128): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-21T17:39:46.236 INFO:teuthology.orchestra.run.smithi003.stdout:(45/128): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-05-21T17:39:46.262 INFO:teuthology.orchestra.run.smithi003.stdout:(46/128): boost-program-options-1.75.0-8.el9.x8 556 kB/s | 107 kB 00:00 2024-05-21T17:39:46.286 INFO:teuthology.orchestra.run.smithi003.stdout:(47/128): python3-ply-3.11-14.el9.noarch.rpm 223 kB/s | 106 kB 00:00 2024-05-21T17:39:46.312 INFO:teuthology.orchestra.run.smithi003.stdout:(48/128): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-21T17:39:46.362 INFO:teuthology.orchestra.run.smithi003.stdout:(49/128): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-05-21T17:39:46.454 INFO:teuthology.orchestra.run.smithi003.stdout:(50/128): libpmemobj-1.12.1-1.el9.x86_64.rpm 957 kB/s | 160 kB 00:00 2024-05-21T17:39:46.504 INFO:teuthology.orchestra.run.smithi003.stdout:(51/128): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-05-21T17:39:46.547 INFO:teuthology.orchestra.run.smithi003.stdout:(52/128): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-21T17:39:46.614 INFO:teuthology.orchestra.run.smithi003.stdout:(53/128): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-21T17:39:46.680 INFO:teuthology.orchestra.run.smithi003.stdout:(54/128): openblas-0.3.21-2.el9.x86_64.rpm 264 kB/s | 35 kB 00:00 2024-05-21T17:39:46.765 INFO:teuthology.orchestra.run.smithi003.stdout:(55/128): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-05-21T17:39:46.876 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T17:39:46.876 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-21T17:39:46.915 INFO:teuthology.orchestra.run.smithi003.stdout:(56/128): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-05-21T17:39:46.937 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-05-21T17:39:46.955 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-05-21T17:39:46.955 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-05-21T17:39:46.956 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-05-21T17:39:46.956 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-05-21T17:39:46.956 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 6.6 k 2024-05-21T17:39:46.956 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 5.4 M 2024-05-21T17:39:46.956 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 890 k 2024-05-21T17:39:46.956 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 145 k 2024-05-21T17:39:46.956 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 1.5 M 2024-05-21T17:39:46.956 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 143 k 2024-05-21T17:39:46.956 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 3.5 M 2024-05-21T17:39:46.956 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 7.4 M 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 49 k 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 11 M 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 49 M 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 273 k 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 764 k 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 32 k 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 747 k 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 127 k 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 162 k 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 321 k 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 300 k 2024-05-21T17:39:46.957 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 99 k 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 85 k 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 3.1 M 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 170 k 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout: sqlite-devel x86_64 3.34.1-7.el9 appstream 134 k 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 3.4 M 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 3.1 M 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-21T17:39:46.958 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 21 M 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 34 k 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 2.4 M 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 274 k 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 4.7 M 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 17 M 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3870.g9ef9bcfd.el9 ceph-noarch 16 k 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 25 k 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-21T17:39:46.959 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 163 k 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 501 k 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-21T17:39:46.960 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 5.4 M 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-21T17:39:46.961 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 45 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:19.0.0-3870.g9ef9bcfd.el9 ceph 136 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-21T17:39:46.962 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-21T17:39:46.963 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-21T17:39:46.964 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-21T17:39:46.965 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-21T17:39:46.966 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:39:46.967 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-05-21T17:39:46.967 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-05-21T17:39:46.967 INFO:teuthology.orchestra.run.smithi203.stdout:Install 126 Packages 2024-05-21T17:39:46.967 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 2 Packages 2024-05-21T17:39:46.967 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:39:46.967 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 203 M 2024-05-21T17:39:46.967 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-05-21T17:39:47.006 INFO:teuthology.orchestra.run.smithi003.stdout:(57/128): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-21T17:39:47.182 INFO:teuthology.orchestra.run.smithi003.stdout:(58/128): python3-jinja2-2.11.3-6.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-05-21T17:39:47.232 INFO:teuthology.orchestra.run.smithi003.stdout:(59/128): python3-jsonpatch-1.21-16.el9.noarch. 515 kB/s | 26 kB 00:00 2024-05-21T17:39:47.282 INFO:teuthology.orchestra.run.smithi003.stdout:(60/128): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-05-21T17:39:47.341 INFO:teuthology.orchestra.run.smithi003.stdout:(61/128): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-21T17:39:47.400 INFO:teuthology.orchestra.run.smithi003.stdout:(62/128): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-05-21T17:39:47.633 INFO:teuthology.orchestra.run.smithi003.stdout:(63/128): python3-babel-2.9.1-2.el9.noarch.rpm 6.9 MB/s | 6.0 MB 00:00 2024-05-21T17:39:47.725 INFO:teuthology.orchestra.run.smithi003.stdout:(64/128): openblas-openmp-0.3.21-2.el9.x86_64.r 4.3 MB/s | 4.7 MB 00:01 2024-05-21T17:39:47.759 INFO:teuthology.orchestra.run.smithi003.stdout:(65/128): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-05-21T17:39:47.809 INFO:teuthology.orchestra.run.smithi003.stdout:(66/128): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-05-21T17:39:47.834 INFO:teuthology.orchestra.run.smithi003.stdout:(67/128): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-05-21T17:39:47.851 INFO:teuthology.orchestra.run.smithi003.stdout:(68/128): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-21T17:39:47.876 INFO:teuthology.orchestra.run.smithi203.stdout:(1/128): ceph-19.0.0-3870.g9ef9bcfd.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-05-21T17:39:47.926 INFO:teuthology.orchestra.run.smithi003.stdout:(69/128): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-05-21T17:39:47.952 INFO:teuthology.orchestra.run.smithi003.stdout:(70/128): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-05-21T17:39:48.002 INFO:teuthology.orchestra.run.smithi003.stdout:(71/128): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-21T17:39:48.027 INFO:teuthology.orchestra.run.smithi003.stdout:(72/128): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-05-21T17:39:48.053 INFO:teuthology.orchestra.run.smithi003.stdout:(73/128): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-21T17:39:48.103 INFO:teuthology.orchestra.run.smithi003.stdout:(74/128): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-05-21T17:39:48.195 INFO:teuthology.orchestra.run.smithi003.stdout:(75/128): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-05-21T17:39:48.218 INFO:teuthology.orchestra.run.smithi203.stdout:(2/128): ceph-fuse-19.0.0-3870.g9ef9bcfd.el9.x8 2.5 MB/s | 890 kB 00:00 2024-05-21T17:39:48.245 INFO:teuthology.orchestra.run.smithi003.stdout:(76/128): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-05-21T17:39:48.301 INFO:teuthology.orchestra.run.smithi203.stdout:(3/128): ceph-immutable-object-cache-19.0.0-387 1.7 MB/s | 145 kB 00:00 2024-05-21T17:39:48.329 INFO:teuthology.orchestra.run.smithi003.stdout:(77/128): qatzip-libs-1.2.0-1.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-05-21T17:39:48.421 INFO:teuthology.orchestra.run.smithi003.stdout:(78/128): socat-1.7.4.1-5.el9.x86_64.rpm 3.3 MB/s | 305 kB 00:00 2024-05-21T17:39:48.496 INFO:teuthology.orchestra.run.smithi003.stdout:(79/128): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.7 MB/s | 5.1 MB 00:01 2024-05-21T17:39:48.555 INFO:teuthology.orchestra.run.smithi003.stdout:(80/128): sqlite-devel-3.34.1-7.el9.x86_64.rpm 2.2 MB/s | 134 kB 00:00 2024-05-21T17:39:48.613 INFO:teuthology.orchestra.run.smithi003.stdout:(81/128): sqlite-3.34.1-7.el9.x86_64.rpm 3.8 MB/s | 748 kB 00:00 2024-05-21T17:39:48.639 INFO:teuthology.orchestra.run.smithi003.stdout:(82/128): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-05-21T17:39:48.664 INFO:teuthology.orchestra.run.smithi003.stdout:(83/128): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-05-21T17:39:48.718 INFO:teuthology.orchestra.run.smithi203.stdout:(4/128): ceph-mds-19.0.0-3870.g9ef9bcfd.el9.x86 5.7 MB/s | 2.4 MB 00:00 2024-05-21T17:39:48.798 INFO:teuthology.orchestra.run.smithi003.stdout:(84/128): protobuf-compiler-3.14.0-13.el9.x86_6 5.3 MB/s | 863 kB 00:00 2024-05-21T17:39:48.902 INFO:teuthology.orchestra.run.smithi203.stdout:(5/128): ceph-base-19.0.0-3870.g9ef9bcfd.el9.x8 4.3 MB/s | 5.4 MB 00:01 2024-05-21T17:39:49.019 INFO:teuthology.orchestra.run.smithi203.stdout:(6/128): ceph-mgr-19.0.0-3870.g9ef9bcfd.el9.x86 5.0 MB/s | 1.5 MB 00:00 2024-05-21T17:39:49.173 INFO:teuthology.orchestra.run.smithi003.stdout:(85/128): abseil-cpp-20211102.0-3.el9.x86_64.rp 1.1 MB/s | 548 kB 00:00 2024-05-21T17:39:49.231 INFO:teuthology.orchestra.run.smithi003.stdout:(86/128): gperftools-libs-2.9.1-2.el9.x86_64.rp 712 kB/s | 309 kB 00:00 2024-05-21T17:39:49.265 INFO:teuthology.orchestra.run.smithi003.stdout:(87/128): grpc-data-1.46.7-10.el9.noarch.rpm 211 kB/s | 19 kB 00:00 2024-05-21T17:39:49.366 INFO:teuthology.orchestra.run.smithi003.stdout:(88/128): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-05-21T17:39:49.458 INFO:teuthology.orchestra.run.smithi003.stdout:(89/128): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-05-21T17:39:49.478 INFO:teuthology.orchestra.run.smithi203.stdout:(7/128): ceph-mon-19.0.0-3870.g9ef9bcfd.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-05-21T17:39:49.558 INFO:teuthology.orchestra.run.smithi003.stdout:(90/128): libunwind-1.6.2-1.el9.x86_64.rpm 675 kB/s | 67 kB 00:00 2024-05-21T17:39:49.625 INFO:teuthology.orchestra.run.smithi003.stdout:(91/128): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-21T17:39:49.650 INFO:teuthology.orchestra.run.smithi003.stdout:(92/128): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-21T17:39:49.734 INFO:teuthology.orchestra.run.smithi003.stdout:(93/128): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-05-21T17:39:49.759 INFO:teuthology.orchestra.run.smithi003.stdout:(94/128): python3-asyncssh-2.13.2-5.el9.noarch. 4.9 MB/s | 548 kB 00:00 2024-05-21T17:39:49.826 INFO:teuthology.orchestra.run.smithi003.stdout:(95/128): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 473 kB/s | 43 kB 00:00 2024-05-21T17:39:49.860 INFO:teuthology.orchestra.run.smithi003.stdout:(96/128): python3-cachetools-4.2.4-1.el9.noarch 320 kB/s | 32 kB 00:00 2024-05-21T17:39:49.918 INFO:teuthology.orchestra.run.smithi003.stdout:(97/128): python3-certifi-2021.10.8-2.el9.noarc 163 kB/s | 15 kB 00:00 2024-05-21T17:39:49.952 INFO:teuthology.orchestra.run.smithi003.stdout:(98/128): python3-cheroot-8.6.0-4.el9.noarch.rp 1.8 MB/s | 172 kB 00:00 2024-05-21T17:39:50.019 INFO:teuthology.orchestra.run.smithi003.stdout:(99/128): python3-cherrypy-18.6.1-2.el9.noarch. 3.5 MB/s | 358 kB 00:00 2024-05-21T17:39:50.053 INFO:teuthology.orchestra.run.smithi003.stdout:(100/128): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-21T17:39:50.144 INFO:teuthology.orchestra.run.smithi003.stdout:(101/128): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-21T17:39:50.228 INFO:teuthology.orchestra.run.smithi003.stdout:(102/128): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-05-21T17:39:50.312 INFO:teuthology.orchestra.run.smithi003.stdout:(103/128): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-05-21T17:39:50.395 INFO:teuthology.orchestra.run.smithi003.stdout:(104/128): python3-jaraco-collections-3.0.0-8.e 278 kB/s | 23 kB 00:00 2024-05-21T17:39:50.479 INFO:teuthology.orchestra.run.smithi003.stdout:(105/128): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-05-21T17:39:50.563 INFO:teuthology.orchestra.run.smithi003.stdout:(106/128): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-05-21T17:39:50.619 INFO:teuthology.orchestra.run.smithi203.stdout:(8/128): ceph-common-19.0.0-3870.g9ef9bcfd.el9. 7.2 MB/s | 21 MB 00:02 2024-05-21T17:39:50.722 INFO:teuthology.orchestra.run.smithi003.stdout:(107/128): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-05-21T17:39:50.745 INFO:teuthology.orchestra.run.smithi203.stdout:(9/128): ceph-radosgw-19.0.0-3870.g9ef9bcfd.el9 8.4 MB/s | 11 MB 00:01 2024-05-21T17:39:50.755 INFO:teuthology.orchestra.run.smithi003.stdout:(108/128): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-05-21T17:39:50.770 INFO:teuthology.orchestra.run.smithi203.stdout:(10/128): ceph-selinux-19.0.0-3870.g9ef9bcfd.el 167 kB/s | 25 kB 00:00 2024-05-21T17:39:50.805 INFO:teuthology.orchestra.run.smithi003.stdout:(109/128): python3-logutils-0.3.5-21.el9.noarch 556 kB/s | 46 kB 00:00 2024-05-21T17:39:50.845 INFO:teuthology.orchestra.run.smithi203.stdout:(11/128): libcephfs-devel-19.0.0-3870.g9ef9bcfd 422 kB/s | 32 kB 00:00 2024-05-21T17:39:50.847 INFO:teuthology.orchestra.run.smithi003.stdout:(110/128): python3-more-itertools-8.12.0-2.el9. 857 kB/s | 79 kB 00:00 2024-05-21T17:39:50.889 INFO:teuthology.orchestra.run.smithi003.stdout:(111/128): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-05-21T17:39:50.948 INFO:teuthology.orchestra.run.smithi003.stdout:(112/128): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-05-21T17:39:50.973 INFO:teuthology.orchestra.run.smithi003.stdout:(113/128): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-05-21T17:39:51.040 INFO:teuthology.orchestra.run.smithi003.stdout:(114/128): python3-pyOpenSSL-21.0.0-1.el9.noarc 979 kB/s | 90 kB 00:00 2024-05-21T17:39:51.065 INFO:teuthology.orchestra.run.smithi003.stdout:(115/128): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-05-21T17:39:51.071 INFO:teuthology.orchestra.run.smithi203.stdout:(12/128): ceph-osd-19.0.0-3870.g9ef9bcfd.el9.x8 8.2 MB/s | 17 MB 00:02 2024-05-21T17:39:51.113 INFO:teuthology.orchestra.run.smithi203.stdout:(13/128): libcephfs2-19.0.0-3870.g9ef9bcfd.el9. 2.7 MB/s | 747 kB 00:00 2024-05-21T17:39:51.142 INFO:teuthology.orchestra.run.smithi003.stdout:(116/128): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-21T17:39:51.154 INFO:teuthology.orchestra.run.smithi203.stdout:(14/128): libcephsqlite-19.0.0-3870.g9ef9bcfd.e 1.9 MB/s | 163 kB 00:00 2024-05-21T17:39:51.166 INFO:teuthology.orchestra.run.smithi003.stdout:(117/128): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-05-21T17:39:51.188 INFO:teuthology.orchestra.run.smithi203.stdout:(15/128): librados-devel-19.0.0-3870.g9ef9bcfd. 1.6 MB/s | 127 kB 00:00 2024-05-21T17:39:51.233 INFO:teuthology.orchestra.run.smithi003.stdout:(118/128): python3-tempora-5.0.0-2.el9.noarch.r 394 kB/s | 36 kB 00:00 2024-05-21T17:39:51.255 INFO:teuthology.orchestra.run.smithi203.stdout:(16/128): libradosstriper1-19.0.0-3870.g9ef9bcf 4.9 MB/s | 501 kB 00:00 2024-05-21T17:39:51.258 INFO:teuthology.orchestra.run.smithi003.stdout:(119/128): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-05-21T17:39:51.333 INFO:teuthology.orchestra.run.smithi003.stdout:(120/128): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-21T17:39:51.356 INFO:teuthology.orchestra.run.smithi203.stdout:(17/128): python3-ceph-argparse-19.0.0-3870.g9e 452 kB/s | 45 kB 00:00 2024-05-21T17:39:51.359 INFO:teuthology.orchestra.run.smithi003.stdout:(121/128): python3-websocket-client-1.2.3-2.el9 894 kB/s | 90 kB 00:00 2024-05-21T17:39:51.434 INFO:teuthology.orchestra.run.smithi003.stdout:(122/128): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-21T17:39:51.459 INFO:teuthology.orchestra.run.smithi003.stdout:(123/128): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-05-21T17:39:51.464 INFO:teuthology.orchestra.run.smithi203.stdout:(18/128): python3-ceph-common-19.0.0-3870.g9ef9 1.2 MB/s | 136 kB 00:00 2024-05-21T17:39:51.526 INFO:teuthology.orchestra.run.smithi003.stdout:(124/128): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-21T17:39:51.581 INFO:teuthology.orchestra.run.smithi203.stdout:(19/128): python3-cephfs-19.0.0-3870.g9ef9bcfd. 1.4 MB/s | 162 kB 00:00 2024-05-21T17:39:51.898 INFO:teuthology.orchestra.run.smithi203.stdout:(20/128): librgw2-19.0.0-3870.g9ef9bcfd.el9.x86 7.6 MB/s | 5.4 MB 00:00 2024-05-21T17:39:52.024 INFO:teuthology.orchestra.run.smithi203.stdout:(21/128): python3-rbd-19.0.0-3870.g9ef9bcfd.el9 2.4 MB/s | 300 kB 00:00 2024-05-21T17:39:52.116 INFO:teuthology.orchestra.run.smithi203.stdout:(22/128): python3-rgw-19.0.0-3870.g9ef9bcfd.el9 1.1 MB/s | 99 kB 00:00 2024-05-21T17:39:52.174 INFO:teuthology.orchestra.run.smithi203.stdout:(23/128): python3-rados-19.0.0-3870.g9ef9bcfd.e 541 kB/s | 321 kB 00:00 2024-05-21T17:39:52.176 INFO:teuthology.orchestra.run.smithi003.stdout:(125/128): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.9 MB/s | 16 MB 00:04 2024-05-21T17:39:52.200 INFO:teuthology.orchestra.run.smithi203.stdout:(24/128): rbd-fuse-19.0.0-3870.g9ef9bcfd.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-21T17:39:52.219 INFO:teuthology.orchestra.run.smithi003.stdout:(126/128): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-05-21T17:39:52.300 INFO:teuthology.orchestra.run.smithi203.stdout:(25/128): rbd-nbd-19.0.0-3870.g9ef9bcfd.el9.x86 1.7 MB/s | 170 kB 00:00 2024-05-21T17:39:52.335 INFO:teuthology.orchestra.run.smithi003.stdout:(127/128): librados2-19.0.0-3870.g9ef9bcfd.el9. 4.2 MB/s | 3.4 MB 00:00 2024-05-21T17:39:52.392 INFO:teuthology.orchestra.run.smithi203.stdout:(26/128): ceph-grafana-dashboards-19.0.0-3870.g 373 kB/s | 34 kB 00:00 2024-05-21T17:39:52.484 INFO:teuthology.orchestra.run.smithi203.stdout:(27/128): ceph-mgr-cephadm-19.0.0-3870.g9ef9bcf 1.5 MB/s | 143 kB 00:00 2024-05-21T17:39:52.809 INFO:teuthology.orchestra.run.smithi203.stdout:(28/128): rbd-mirror-19.0.0-3870.g9ef9bcfd.el9. 4.9 MB/s | 3.1 MB 00:00 2024-05-21T17:39:52.985 INFO:teuthology.orchestra.run.smithi003.stdout:(128/128): librbd1-19.0.0-3870.g9ef9bcfd.el9.x8 3.9 MB/s | 3.1 MB 00:00 2024-05-21T17:39:52.988 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-05-21T17:39:52.989 INFO:teuthology.orchestra.run.smithi003.stdout:Total 14 MB/s | 203 MB 00:14 2024-05-21T17:39:53.726 INFO:teuthology.orchestra.run.smithi203.stdout:(29/128): ceph-mgr-dashboard-19.0.0-3870.g9ef9b 2.9 MB/s | 3.5 MB 00:01 2024-05-21T17:39:53.845 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-05-21T17:39:53.902 INFO:teuthology.orchestra.run.smithi203.stdout:(30/128): ceph-mgr-modules-core-19.0.0-3870.g9e 1.5 MB/s | 274 kB 00:00 2024-05-21T17:39:53.958 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-05-21T17:39:53.959 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-05-21T17:39:53.977 INFO:teuthology.orchestra.run.smithi203.stdout:(31/128): ceph-mgr-rook-19.0.0-3870.g9ef9bcfd.e 658 kB/s | 49 kB 00:00 2024-05-21T17:39:54.044 INFO:teuthology.orchestra.run.smithi203.stdout:(32/128): ceph-prometheus-alerts-19.0.0-3870.g9 247 kB/s | 16 kB 00:00 2024-05-21T17:39:54.203 INFO:teuthology.orchestra.run.smithi203.stdout:(33/128): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-05-21T17:39:54.228 INFO:teuthology.orchestra.run.smithi203.stdout:(34/128): ceph-volume-19.0.0-3870.g9ef9bcfd.el9 1.5 MB/s | 273 kB 00:00 2024-05-21T17:39:54.287 INFO:teuthology.orchestra.run.smithi203.stdout:(35/128): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 80 kB 00:00 2024-05-21T17:39:54.379 INFO:teuthology.orchestra.run.smithi203.stdout:(36/128): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-05-21T17:39:54.454 INFO:teuthology.orchestra.run.smithi203.stdout:(37/128): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-05-21T17:39:54.529 INFO:teuthology.orchestra.run.smithi203.stdout:(38/128): cephadm-19.0.0-3870.g9ef9bcfd.el9.noa 2.3 MB/s | 764 kB 00:00 2024-05-21T17:39:54.563 INFO:teuthology.orchestra.run.smithi203.stdout:(39/128): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-05-21T17:39:54.630 INFO:teuthology.orchestra.run.smithi203.stdout:(40/128): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-21T17:39:54.689 INFO:teuthology.orchestra.run.smithi203.stdout:(41/128): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.2 MB 00:00 2024-05-21T17:39:54.722 INFO:teuthology.orchestra.run.smithi203.stdout:(42/128): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-05-21T17:39:54.781 INFO:teuthology.orchestra.run.smithi203.stdout:(43/128): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-21T17:39:54.823 INFO:teuthology.orchestra.run.smithi203.stdout:(44/128): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-05-21T17:39:55.048 INFO:teuthology.orchestra.run.smithi203.stdout:(45/128): flexiblas-3.0.4-8.el9.x86_64.rpm 145 kB/s | 33 kB 00:00 2024-05-21T17:39:55.124 INFO:teuthology.orchestra.run.smithi203.stdout:(46/128): boost-program-options-1.75.0-8.el9.x8 312 kB/s | 107 kB 00:00 2024-05-21T17:39:55.207 INFO:teuthology.orchestra.run.smithi203.stdout:(47/128): flexiblas-openblas-openmp-3.0.4-8.el9 218 kB/s | 18 kB 00:00 2024-05-21T17:39:55.383 INFO:teuthology.orchestra.run.smithi203.stdout:(48/128): libpmemobj-1.12.1-1.el9.x86_64.rpm 915 kB/s | 160 kB 00:00 2024-05-21T17:39:55.466 INFO:teuthology.orchestra.run.smithi203.stdout:(49/128): librabbitmq-0.11.0-7.el9.x86_64.rpm 544 kB/s | 45 kB 00:00 2024-05-21T17:39:55.675 INFO:teuthology.orchestra.run.smithi203.stdout:(50/128): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-05-21T17:39:55.767 INFO:teuthology.orchestra.run.smithi203.stdout:(51/128): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-21T17:39:55.855 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-05-21T17:39:55.856 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-05-21T17:39:55.868 INFO:teuthology.orchestra.run.smithi203.stdout:(52/128): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-21T17:39:56.326 INFO:teuthology.orchestra.run.smithi203.stdout:(53/128): ceph-test-19.0.0-3870.g9ef9bcfd.el9.x 8.8 MB/s | 49 MB 00:05 2024-05-21T17:39:56.345 INFO:teuthology.orchestra.run.smithi203.stdout:(54/128): openblas-0.3.21-2.el9.x86_64.rpm 74 kB/s | 35 kB 00:00 2024-05-21T17:39:56.518 INFO:teuthology.orchestra.run.smithi203.stdout:(55/128): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.0 MB/s | 3.0 MB 00:01 2024-05-21T17:39:56.627 INFO:teuthology.orchestra.run.smithi203.stdout:(56/128): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-05-21T17:39:56.719 INFO:teuthology.orchestra.run.smithi203.stdout:(57/128): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-05-21T17:39:56.836 INFO:teuthology.orchestra.run.smithi203.stdout:(58/128): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-05-21T17:39:56.895 INFO:teuthology.orchestra.run.smithi203.stdout:(59/128): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-05-21T17:39:56.953 INFO:teuthology.orchestra.run.smithi203.stdout:(60/128): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-05-21T17:39:57.045 INFO:teuthology.orchestra.run.smithi203.stdout:(61/128): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-05-21T17:39:57.104 INFO:teuthology.orchestra.run.smithi203.stdout:(62/128): python3-markupsafe-1.1.1-12.el9.x86_6 596 kB/s | 35 kB 00:00 2024-05-21T17:39:57.238 INFO:teuthology.orchestra.run.smithi203.stdout:(63/128): openblas-openmp-0.3.21-2.el9.x86_64.r 5.2 MB/s | 4.7 MB 00:00 2024-05-21T17:39:57.305 INFO:teuthology.orchestra.run.smithi203.stdout:(64/128): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-05-21T17:39:57.363 INFO:teuthology.orchestra.run.smithi203.stdout:(65/128): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-21T17:39:57.422 INFO:teuthology.orchestra.run.smithi203.stdout:(66/128): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-21T17:39:57.481 INFO:teuthology.orchestra.run.smithi203.stdout:(67/128): python3-prettytable-0.7.2-27.el9.noar 725 kB/s | 42 kB 00:00 2024-05-21T17:39:57.539 INFO:teuthology.orchestra.run.smithi203.stdout:(68/128): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-05-21T17:39:57.598 INFO:teuthology.orchestra.run.smithi203.stdout:(69/128): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-21T17:39:57.656 INFO:teuthology.orchestra.run.smithi203.stdout:(70/128): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-05-21T17:39:57.716 INFO:teuthology.orchestra.run.smithi203.stdout:(71/128): python3-pytz-2021.1-5.el9.noarch.rpm 867 kB/s | 51 kB 00:00 2024-05-21T17:39:57.774 INFO:teuthology.orchestra.run.smithi203.stdout:(72/128): python3-requests-oauthlib-1.3.0-12.el 921 kB/s | 54 kB 00:00 2024-05-21T17:39:58.416 INFO:teuthology.orchestra.run.smithi203.stdout:(73/128): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.9 MB/s | 5.1 MB 00:01 2024-05-21T17:39:58.475 INFO:teuthology.orchestra.run.smithi203.stdout:(74/128): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-05-21T17:39:58.567 INFO:teuthology.orchestra.run.smithi203.stdout:(75/128): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-05-21T17:39:58.734 INFO:teuthology.orchestra.run.smithi203.stdout:(76/128): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-05-21T17:39:58.759 INFO:teuthology.orchestra.run.smithi203.stdout:(77/128): qatlib-service-24.02.0-1.el9.x86_64.r 187 kB/s | 36 kB 00:00 2024-05-21T17:39:58.835 INFO:teuthology.orchestra.run.smithi203.stdout:(78/128): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-21T17:39:59.002 INFO:teuthology.orchestra.run.smithi203.stdout:(79/128): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-05-21T17:39:59.177 INFO:teuthology.orchestra.run.smithi203.stdout:(80/128): python3-babel-2.9.1-2.el9.noarch.rpm 2.2 MB/s | 6.0 MB 00:02 2024-05-21T17:39:59.202 INFO:teuthology.orchestra.run.smithi203.stdout:(81/128): sqlite-3.34.1-7.el9.x86_64.rpm 2.0 MB/s | 748 kB 00:00 2024-05-21T17:39:59.228 INFO:teuthology.orchestra.run.smithi203.stdout:(82/128): sqlite-devel-3.34.1-7.el9.x86_64.rpm 594 kB/s | 134 kB 00:00 2024-05-21T17:39:59.261 INFO:teuthology.orchestra.run.smithi203.stdout:(83/128): xmlstarlet-1.6.1-20.el9.x86_64.rpm 760 kB/s | 64 kB 00:00 2024-05-21T17:39:59.278 INFO:teuthology.orchestra.run.smithi203.stdout:(84/128): lua-devel-5.4.4-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2024-05-21T17:39:59.362 INFO:teuthology.orchestra.run.smithi203.stdout:(85/128): protobuf-compiler-3.14.0-13.el9.x86_6 6.3 MB/s | 863 kB 00:00 2024-05-21T17:39:59.529 INFO:teuthology.orchestra.run.smithi203.stdout:(86/128): grpc-data-1.46.7-10.el9.noarch.rpm 117 kB/s | 19 kB 00:00 2024-05-21T17:39:59.613 INFO:teuthology.orchestra.run.smithi203.stdout:(87/128): gperftools-libs-2.9.1-2.el9.x86_64.rp 925 kB/s | 309 kB 00:00 2024-05-21T17:39:59.655 INFO:teuthology.orchestra.run.smithi203.stdout:(88/128): abseil-cpp-20211102.0-3.el9.x86_64.rp 1.4 MB/s | 548 kB 00:00 2024-05-21T17:39:59.688 INFO:teuthology.orchestra.run.smithi203.stdout:(89/128): libarrow-doc-9.0.0-11.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-05-21T17:39:59.738 INFO:teuthology.orchestra.run.smithi203.stdout:(90/128): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-05-21T17:39:59.755 INFO:teuthology.orchestra.run.smithi203.stdout:(91/128): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-21T17:39:59.856 INFO:teuthology.orchestra.run.smithi203.stdout:(92/128): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-05-21T17:39:59.956 INFO:teuthology.orchestra.run.smithi203.stdout:(93/128): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-05-21T17:40:00.015 INFO:teuthology.orchestra.run.smithi203.stdout:(94/128): python3-asyncssh-2.13.2-5.el9.noarch. 3.4 MB/s | 548 kB 00:00 2024-05-21T17:40:00.040 INFO:teuthology.orchestra.run.smithi203.stdout:(95/128): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 517 kB/s | 43 kB 00:00 2024-05-21T17:40:00.116 INFO:teuthology.orchestra.run.smithi203.stdout:(96/128): libarrow-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 4.4 MB 00:00 2024-05-21T17:40:00.141 INFO:teuthology.orchestra.run.smithi203.stdout:(97/128): python3-certifi-2021.10.8-2.el9.noarc 149 kB/s | 15 kB 00:00 2024-05-21T17:40:00.166 INFO:teuthology.orchestra.run.smithi203.stdout:(98/128): python3-cachetools-4.2.4-1.el9.noarch 213 kB/s | 32 kB 00:00 2024-05-21T17:40:00.192 INFO:teuthology.orchestra.run.smithi203.stdout:(99/128): python3-cheroot-8.6.0-4.el9.noarch.rp 2.2 MB/s | 172 kB 00:00 2024-05-21T17:40:00.225 INFO:teuthology.orchestra.run.smithi203.stdout:(100/128): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-21T17:40:00.251 INFO:teuthology.orchestra.run.smithi203.stdout:(101/128): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-21T17:40:00.284 INFO:teuthology.orchestra.run.smithi203.stdout:(102/128): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-21T17:40:00.334 INFO:teuthology.orchestra.run.smithi203.stdout:(103/128): python3-jaraco-8.2.1-3.el9.noarch.rp 127 kB/s | 11 kB 00:00 2024-05-21T17:40:00.377 INFO:teuthology.orchestra.run.smithi203.stdout:(104/128): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-21T17:40:00.402 INFO:teuthology.orchestra.run.smithi203.stdout:(105/128): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-05-21T17:40:00.427 INFO:teuthology.orchestra.run.smithi203.stdout:(106/128): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-05-21T17:40:00.453 INFO:teuthology.orchestra.run.smithi203.stdout:(107/128): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-05-21T17:40:00.478 INFO:teuthology.orchestra.run.smithi203.stdout:(108/128): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-21T17:40:00.520 INFO:teuthology.orchestra.run.smithi203.stdout:(109/128): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-05-21T17:40:00.545 INFO:teuthology.orchestra.run.smithi203.stdout:(110/128): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-05-21T17:40:00.570 INFO:teuthology.orchestra.run.smithi203.stdout:(111/128): python3-more-itertools-8.12.0-2.el9. 854 kB/s | 79 kB 00:00 2024-05-21T17:40:00.596 INFO:teuthology.orchestra.run.smithi203.stdout:(112/128): python3-natsort-7.1.1-5.el9.noarch.r 763 kB/s | 58 kB 00:00 2024-05-21T17:40:00.621 INFO:teuthology.orchestra.run.smithi203.stdout:(113/128): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-21T17:40:00.646 INFO:teuthology.orchestra.run.smithi203.stdout:(114/128): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-05-21T17:40:00.672 INFO:teuthology.orchestra.run.smithi203.stdout:(115/128): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-21T17:40:00.697 INFO:teuthology.orchestra.run.smithi203.stdout:(116/128): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-21T17:40:00.722 INFO:teuthology.orchestra.run.smithi203.stdout:(117/128): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-21T17:40:00.748 INFO:teuthology.orchestra.run.smithi203.stdout:(118/128): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-05-21T17:40:00.764 INFO:teuthology.orchestra.run.smithi203.stdout:(119/128): python3-rsa-4.9-2.el9.noarch.rpm 639 kB/s | 59 kB 00:00 2024-05-21T17:40:00.790 INFO:teuthology.orchestra.run.smithi203.stdout:(120/128): python3-typing-extensions-4.4.0-2.el 758 kB/s | 51 kB 00:00 2024-05-21T17:40:00.815 INFO:teuthology.orchestra.run.smithi203.stdout:(121/128): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-05-21T17:40:00.841 INFO:teuthology.orchestra.run.smithi203.stdout:(122/128): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-21T17:40:00.866 INFO:teuthology.orchestra.run.smithi203.stdout:(123/128): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-05-21T17:40:00.891 INFO:teuthology.orchestra.run.smithi203.stdout:(124/128): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-05-21T17:40:00.911 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-05-21T17:40:00.916 INFO:teuthology.orchestra.run.smithi203.stdout:(125/128): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-21T17:40:00.981 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/130 2024-05-21T17:40:01.000 INFO:teuthology.orchestra.run.smithi203.stdout:(126/128): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-21T17:40:01.006 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/130 2024-05-21T17:40:01.076 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/130 2024-05-21T17:40:01.417 INFO:teuthology.orchestra.run.smithi203.stdout:(127/128): librbd1-19.0.0-3870.g9ef9bcfd.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-05-21T17:40:01.474 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/130 2024-05-21T17:40:01.532 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 5/130 2024-05-21T17:40:01.634 INFO:teuthology.orchestra.run.smithi203.stdout:(128/128): librados2-19.0.0-3870.g9ef9bcfd.el9. 4.6 MB/s | 3.4 MB 00:00 2024-05-21T17:40:01.637 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-05-21T17:40:01.637 INFO:teuthology.orchestra.run.smithi203.stdout:Total 14 MB/s | 203 MB 00:14 2024-05-21T17:40:01.695 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 5/130 2024-05-21T17:40:01.733 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 6/130 2024-05-21T17:40:01.838 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 6/130 2024-05-21T17:40:01.885 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 7/130 2024-05-21T17:40:01.912 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/130 2024-05-21T17:40:01.949 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/130 2024-05-21T17:40:01.977 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 10/130 2024-05-21T17:40:02.083 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 10/130 2024-05-21T17:40:02.112 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:19.0.0-3870.g9ef9bcfd.el9.x86 11/130 2024-05-21T17:40:02.175 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3870.g9ef9bcfd.el9.x86 11/130 2024-05-21T17:40:02.204 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-05-21T17:40:02.231 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-05-21T17:40:02.415 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 13/130 2024-05-21T17:40:02.446 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 13/130 2024-05-21T17:40:02.468 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-05-21T17:40:02.559 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : re2-1:20211101-3.el9.x86_64 14/130 2024-05-21T17:40:02.571 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-05-21T17:40:02.571 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-05-21T17:40:02.594 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/130 2024-05-21T17:40:02.638 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.7-2.el9.x86_64 16/130 2024-05-21T17:40:02.706 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 17/130 2024-05-21T17:40:02.754 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 18/130 2024-05-21T17:40:02.797 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/130 2024-05-21T17:40:02.830 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 20/130 2024-05-21T17:40:02.866 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-5.4.4-4.el9.x86_64 21/130 2024-05-21T17:40:02.938 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/130 2024-05-21T17:40:02.998 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 23/130 2024-05-21T17:40:03.033 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/130 2024-05-21T17:40:03.073 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/130 2024-05-21T17:40:03.151 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/130 2024-05-21T17:40:03.182 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:19.0.0-3870.g9ef9bcfd.el9. 27/130 2024-05-21T17:40:03.217 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:19.0.0-3870.g9ef9bcfd.el 28/130 2024-05-21T17:40:03.245 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:19.0.0-3870.g9ef9bcfd.el9.x86_6 29/130 2024-05-21T17:40:03.320 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 30/130 2024-05-21T17:40:03.369 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : luarocks-3.9.2-5.el9.noarch 31/130 2024-05-21T17:40:03.447 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 32/130 2024-05-21T17:40:03.593 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/130 2024-05-21T17:40:03.656 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/130 2024-05-21T17:40:04.217 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/130 2024-05-21T17:40:04.304 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/130 2024-05-21T17:40:04.338 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 37/130 2024-05-21T17:40:04.380 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 38/130 2024-05-21T17:40:04.415 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-05-21T17:40:04.416 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-05-21T17:40:04.418 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_6 39/130 2024-05-21T17:40:04.443 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 40/130 2024-05-21T17:40:04.538 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/130 2024-05-21T17:40:04.609 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 42/130 2024-05-21T17:40:04.683 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 43/130 2024-05-21T17:40:04.719 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 44/130 2024-05-21T17:40:04.756 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 45/130 2024-05-21T17:40:04.804 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 46/130 2024-05-21T17:40:04.849 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 47/130 2024-05-21T17:40:04.891 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 48/130 2024-05-21T17:40:05.035 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 49/130 2024-05-21T17:40:05.084 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 50/130 2024-05-21T17:40:05.125 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 51/130 2024-05-21T17:40:05.224 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 52/130 2024-05-21T17:40:05.258 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 53/130 2024-05-21T17:40:05.300 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 54/130 2024-05-21T17:40:05.332 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 55/130 2024-05-21T17:40:05.358 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 56/130 2024-05-21T17:40:05.453 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 57/130 2024-05-21T17:40:05.590 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 58/130 2024-05-21T17:40:05.633 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 59/130 2024-05-21T17:40:05.678 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 60/130 2024-05-21T17:40:05.721 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 61/130 2024-05-21T17:40:05.756 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 62/130 2024-05-21T17:40:05.785 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 63/130 2024-05-21T17:40:05.820 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 64/130 2024-05-21T17:40:05.853 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 65/130 2024-05-21T17:40:05.895 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 66/130 2024-05-21T17:40:05.937 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 67/130 2024-05-21T17:40:06.028 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 68/130 2024-05-21T17:40:06.077 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 69/130 2024-05-21T17:40:06.107 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 70/130 2024-05-21T17:40:06.155 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 71/130 2024-05-21T17:40:06.268 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-packaging-20.9-5.el9.noarch 72/130 2024-05-21T17:40:06.318 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 73/130 2024-05-21T17:40:07.096 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-05-21T17:40:07.149 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 75/130 2024-05-21T17:40:07.183 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 76/130 2024-05-21T17:40:07.291 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 77/130 2024-05-21T17:40:07.325 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 78/130 2024-05-21T17:40:07.510 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.21-2.el9.x86_64 79/130 2024-05-21T17:40:07.547 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 80/130 2024-05-21T17:40:07.629 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 81/130 2024-05-21T17:40:08.332 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 82/130 2024-05-21T17:40:08.452 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 83/130 2024-05-21T17:40:09.655 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-05-21T17:40:09.730 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/130 2024-05-21T17:40:09.755 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/130 2024-05-21T17:40:09.807 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/130 2024-05-21T17:40:09.808 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/130 2024-05-21T17:40:09.888 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 85/130 2024-05-21T17:40:10.164 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/130 2024-05-21T17:40:10.218 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 86/130 2024-05-21T17:40:10.249 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 87/130 2024-05-21T17:40:10.281 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 5/130 2024-05-21T17:40:10.333 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 87/130 2024-05-21T17:40:10.362 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 88/130 2024-05-21T17:40:10.386 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 5/130 2024-05-21T17:40:10.407 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 89/130 2024-05-21T17:40:10.423 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 6/130 2024-05-21T17:40:10.499 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 6/130 2024-05-21T17:40:10.541 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 7/130 2024-05-21T17:40:10.570 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/130 2024-05-21T17:40:10.607 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/130 2024-05-21T17:40:10.635 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 10/130 2024-05-21T17:40:10.722 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 10/130 2024-05-21T17:40:10.744 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:19.0.0-3870.g9ef9bcfd.el9.x86 11/130 2024-05-21T17:40:10.807 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3870.g9ef9bcfd.el9.x86 11/130 2024-05-21T17:40:10.829 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-05-21T17:40:10.854 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-05-21T17:40:10.905 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 90/130 2024-05-21T17:40:10.935 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 91/130 2024-05-21T17:40:10.999 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 91/130 2024-05-21T17:40:11.028 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 92/130 2024-05-21T17:40:11.047 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 13/130 2024-05-21T17:40:11.079 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 13/130 2024-05-21T17:40:11.188 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-3.el9.x86_64 14/130 2024-05-21T17:40:11.219 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/130 2024-05-21T17:40:11.254 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 16/130 2024-05-21T17:40:11.319 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 17/130 2024-05-21T17:40:11.363 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 18/130 2024-05-21T17:40:11.406 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/130 2024-05-21T17:40:11.439 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 20/130 2024-05-21T17:40:11.466 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-5.4.4-4.el9.x86_64 21/130 2024-05-21T17:40:11.536 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/130 2024-05-21T17:40:11.590 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 23/130 2024-05-21T17:40:11.623 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/130 2024-05-21T17:40:11.664 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/130 2024-05-21T17:40:11.741 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/130 2024-05-21T17:40:11.774 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:19.0.0-3870.g9ef9bcfd.el9. 27/130 2024-05-21T17:40:11.818 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:19.0.0-3870.g9ef9bcfd.el 28/130 2024-05-21T17:40:11.847 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:19.0.0-3870.g9ef9bcfd.el9.x86_6 29/130 2024-05-21T17:40:11.919 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 30/130 2024-05-21T17:40:11.962 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : luarocks-3.9.2-5.el9.noarch 31/130 2024-05-21T17:40:12.037 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 32/130 2024-05-21T17:40:12.181 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/130 2024-05-21T17:40:12.242 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/130 2024-05-21T17:40:12.776 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/130 2024-05-21T17:40:12.855 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/130 2024-05-21T17:40:12.891 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 37/130 2024-05-21T17:40:12.934 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 38/130 2024-05-21T17:40:12.972 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_6 39/130 2024-05-21T17:40:13.005 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 40/130 2024-05-21T17:40:13.098 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/130 2024-05-21T17:40:13.161 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 42/130 2024-05-21T17:40:13.234 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 43/130 2024-05-21T17:40:13.265 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 44/130 2024-05-21T17:40:13.302 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 45/130 2024-05-21T17:40:13.349 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 46/130 2024-05-21T17:40:13.394 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 47/130 2024-05-21T17:40:13.413 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 93/130 2024-05-21T17:40:13.436 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 48/130 2024-05-21T17:40:13.451 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 93/130 2024-05-21T17:40:13.576 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 49/130 2024-05-21T17:40:13.621 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 50/130 2024-05-21T17:40:13.662 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 51/130 2024-05-21T17:40:13.758 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 52/130 2024-05-21T17:40:13.787 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 53/130 2024-05-21T17:40:13.829 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 54/130 2024-05-21T17:40:13.861 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 55/130 2024-05-21T17:40:13.886 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 56/130 2024-05-21T17:40:13.982 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 57/130 2024-05-21T17:40:14.044 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 93/130 2024-05-21T17:40:14.087 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 94/130 2024-05-21T17:40:14.105 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 58/130 2024-05-21T17:40:14.138 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 59/130 2024-05-21T17:40:14.179 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 60/130 2024-05-21T17:40:14.180 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 94/130 2024-05-21T17:40:14.180 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T17:40:14.180 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T17:40:14.180 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:14.214 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 95/130 2024-05-21T17:40:14.241 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 61/130 2024-05-21T17:40:14.276 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 62/130 2024-05-21T17:40:14.306 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 63/130 2024-05-21T17:40:14.340 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 64/130 2024-05-21T17:40:14.374 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 65/130 2024-05-21T17:40:14.416 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 66/130 2024-05-21T17:40:14.450 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 67/130 2024-05-21T17:40:14.538 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 68/130 2024-05-21T17:40:14.581 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 69/130 2024-05-21T17:40:14.620 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 70/130 2024-05-21T17:40:14.667 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 71/130 2024-05-21T17:40:14.769 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-packaging-20.9-5.el9.noarch 72/130 2024-05-21T17:40:14.821 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 73/130 2024-05-21T17:40:15.551 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-05-21T17:40:15.611 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 75/130 2024-05-21T17:40:15.644 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 76/130 2024-05-21T17:40:15.745 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 77/130 2024-05-21T17:40:15.786 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 78/130 2024-05-21T17:40:16.198 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.21-2.el9.x86_64 79/130 2024-05-21T17:40:16.234 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 80/130 2024-05-21T17:40:16.313 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 81/130 2024-05-21T17:40:16.986 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 82/130 2024-05-21T17:40:17.101 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 83/130 2024-05-21T17:40:18.392 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/130 2024-05-21T17:40:18.482 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 85/130 2024-05-21T17:40:18.804 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 86/130 2024-05-21T17:40:18.844 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 87/130 2024-05-21T17:40:18.935 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 87/130 2024-05-21T17:40:18.965 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 88/130 2024-05-21T17:40:19.008 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 89/130 2024-05-21T17:40:19.490 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 90/130 2024-05-21T17:40:19.513 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 91/130 2024-05-21T17:40:19.568 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 91/130 2024-05-21T17:40:19.590 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 92/130 2024-05-21T17:40:21.875 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 93/130 2024-05-21T17:40:21.911 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 93/130 2024-05-21T17:40:22.463 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 93/130 2024-05-21T17:40:22.498 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 94/130 2024-05-21T17:40:22.573 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 94/130 2024-05-21T17:40:22.574 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-21T17:40:22.574 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-21T17:40:22.574 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:22.608 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 95/130 2024-05-21T17:40:26.064 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 95/130 2024-05-21T17:40:26.065 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2024-05-21T17:40:26.065 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2024-05-21T17:40:26.065 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2024-05-21T17:40:26.065 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2024-05-21T17:40:26.065 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2024-05-21T17:40:26.065 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /root 2024-05-21T17:40:26.065 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2024-05-21T17:40:26.065 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:26.243 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:19.0.0-3870.g9ef9bcfd.el9.noarch 96/130 2024-05-21T17:40:26.261 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:19.0.0-3870.g9ef9bcfd.el9.noarch 96/130 2024-05-21T17:40:26.261 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:26.261 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T17:40:26.261 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:27.806 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 97/130 2024-05-21T17:40:27.834 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 97/130 2024-05-21T17:40:27.834 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:27.835 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T17:40:27.835 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T17:40:27.835 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T17:40:27.835 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:28.073 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 98/130 2024-05-21T17:40:28.098 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 98/130 2024-05-21T17:40:28.098 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:28.098 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T17:40:28.098 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T17:40:28.099 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T17:40:28.099 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:28.574 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 99/130 2024-05-21T17:40:28.599 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 99/130 2024-05-21T17:40:28.599 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:28.599 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T17:40:28.599 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T17:40:28.599 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T17:40:28.600 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:28.670 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ply-3.11-14.el9.noarch 100/130 2024-05-21T17:40:28.742 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 101/130 2024-05-21T17:40:28.938 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 102/130 2024-05-21T17:40:28.999 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 103/130 2024-05-21T17:40:29.080 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 104/130 2024-05-21T17:40:29.168 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 105/130 2024-05-21T17:40:29.320 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 106/130 2024-05-21T17:40:29.358 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 107/130 2024-05-21T17:40:29.389 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 108/130 2024-05-21T17:40:29.416 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.9-15.el9.x86_64 109/130 2024-05-21T17:40:29.839 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noarch 110/130 2024-05-21T17:40:29.865 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noarch 110/130 2024-05-21T17:40:29.898 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3870.g9ef9bcfd.e 111/130 2024-05-21T17:40:29.961 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3870.g9ef9bcfd. 112/130 2024-05-21T17:40:29.988 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noa 113/130 2024-05-21T17:40:31.088 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noa 113/130 2024-05-21T17:40:31.123 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3870.g9ef9bcfd.el9.n 114/130 2024-05-21T17:40:32.137 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3870.g9ef9bcfd.el9.n 114/130 2024-05-21T17:40:32.165 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3870.g9ef 115/130 2024-05-21T17:40:32.283 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3870.g9ef 115/130 2024-05-21T17:40:32.459 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3870.g9ef9bcfd.el 116/130 2024-05-21T17:40:32.492 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 117/130 2024-05-21T17:40:32.518 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 117/130 2024-05-21T17:40:32.518 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:32.518 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T17:40:32.518 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T17:40:32.518 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T17:40:32.518 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:32.568 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:19.0.0-3870.g9ef9bcfd.el9.noarch 118/130 2024-05-21T17:40:32.581 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3870.g9ef9bcfd.el9.noarch 118/130 2024-05-21T17:40:32.681 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 119/130 2024-05-21T17:40:33.666 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 120/130 2024-05-21T17:40:33.695 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 121/130 2024-05-21T17:40:33.720 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 121/130 2024-05-21T17:40:33.720 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:33.720 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T17:40:33.720 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T17:40:33.720 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T17:40:33.720 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:33.762 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3870.g9ef9b 122/130 2024-05-21T17:40:33.786 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3870.g9ef9b 122/130 2024-05-21T17:40:33.787 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:33.787 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T17:40:33.787 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:33.947 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 95/130 2024-05-21T17:40:33.947 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-05-21T17:40:33.947 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-05-21T17:40:33.948 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-05-21T17:40:33.948 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-05-21T17:40:33.948 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-05-21T17:40:33.948 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-05-21T17:40:33.948 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-05-21T17:40:33.948 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:34.112 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 123/130 2024-05-21T17:40:34.121 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:19.0.0-3870.g9ef9bcfd.el9.noarch 96/130 2024-05-21T17:40:34.137 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 123/130 2024-05-21T17:40:34.137 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:34.137 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T17:40:34.137 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T17:40:34.138 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T17:40:34.138 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:34.138 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:19.0.0-3870.g9ef9bcfd.el9.noarch 96/130 2024-05-21T17:40:34.138 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:34.138 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-21T17:40:34.138 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:35.649 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 97/130 2024-05-21T17:40:35.676 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 97/130 2024-05-21T17:40:35.676 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:35.676 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-21T17:40:35.676 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T17:40:35.676 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-21T17:40:35.676 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:35.917 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 98/130 2024-05-21T17:40:35.940 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 98/130 2024-05-21T17:40:35.941 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:35.941 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-21T17:40:35.941 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T17:40:35.941 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-21T17:40:35.941 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:36.384 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 99/130 2024-05-21T17:40:36.408 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 99/130 2024-05-21T17:40:36.408 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:36.408 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-21T17:40:36.408 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T17:40:36.408 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-21T17:40:36.408 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:36.479 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 100/130 2024-05-21T17:40:36.534 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 101/130 2024-05-21T17:40:36.719 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 102/130 2024-05-21T17:40:36.775 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 103/130 2024-05-21T17:40:36.845 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 104/130 2024-05-21T17:40:36.934 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 105/130 2024-05-21T17:40:37.076 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 106/130 2024-05-21T17:40:37.117 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 107/130 2024-05-21T17:40:37.149 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 108/130 2024-05-21T17:40:37.176 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-15.el9.x86_64 109/130 2024-05-21T17:40:37.630 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noarch 110/130 2024-05-21T17:40:37.658 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noarch 110/130 2024-05-21T17:40:37.691 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3870.g9ef9bcfd.e 111/130 2024-05-21T17:40:37.752 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3870.g9ef9bcfd. 112/130 2024-05-21T17:40:37.782 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noa 113/130 2024-05-21T17:40:38.831 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noa 113/130 2024-05-21T17:40:38.867 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3870.g9ef9bcfd.el9.n 114/130 2024-05-21T17:40:39.000 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 124/130 2024-05-21T17:40:39.042 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 125/130 2024-05-21T17:40:39.114 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 126/130 2024-05-21T17:40:39.139 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : sqlite-devel-3.34.1-7.el9.x86_64 127/130 2024-05-21T17:40:39.170 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_ 128/130 2024-05-21T17:40:39.170 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-05-21T17:40:39.263 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-05-21T17:40:39.263 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 130/130 2024-05-21T17:40:39.841 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3870.g9ef9bcfd.el9.n 114/130 2024-05-21T17:40:39.884 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3870.g9ef 115/130 2024-05-21T17:40:39.996 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3870.g9ef 115/130 2024-05-21T17:40:40.162 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3870.g9ef9bcfd.el 116/130 2024-05-21T17:40:40.194 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 117/130 2024-05-21T17:40:40.219 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 117/130 2024-05-21T17:40:40.219 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:40.219 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-21T17:40:40.219 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T17:40:40.219 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-21T17:40:40.219 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:40.270 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:19.0.0-3870.g9ef9bcfd.el9.noarch 118/130 2024-05-21T17:40:40.283 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3870.g9ef9bcfd.el9.noarch 118/130 2024-05-21T17:40:40.379 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 119/130 2024-05-21T17:40:41.303 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 120/130 2024-05-21T17:40:41.338 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 121/130 2024-05-21T17:40:41.362 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 121/130 2024-05-21T17:40:41.362 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:41.362 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-21T17:40:41.362 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T17:40:41.362 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-21T17:40:41.362 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:41.405 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3870.g9ef9b 122/130 2024-05-21T17:40:41.429 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3870.g9ef9b 122/130 2024-05-21T17:40:41.429 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:41.429 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-21T17:40:41.429 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:41.731 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 123/130 2024-05-21T17:40:41.754 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 123/130 2024-05-21T17:40:41.754 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-21T17:40:41.754 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-21T17:40:41.754 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T17:40:41.754 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-21T17:40:41.755 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:44.595 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 130/130 2024-05-21T17:40:44.595 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 1/130 2024-05-21T17:40:44.595 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2/130 2024-05-21T17:40:44.595 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 3/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 4/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3870.g9ef9b 5/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 6/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 7/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 8/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 9/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 10/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 11/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 12/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_ 13/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 14/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 15/130 2024-05-21T17:40:44.596 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_6 16/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:19.0.0-3870.g9ef9bcfd.el9.x86 17/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 18/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3870.g9ef9bcfd.el 19/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:19.0.0-3870.g9ef9bcfd.el9. 20/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:19.0.0-3870.g9ef9bcfd.el9.x86_6 21/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 22/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 23/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 24/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 25/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 26/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 27/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3870.g9ef9bcfd. 28/130 2024-05-21T17:40:44.597 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noa 29/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3870.g9ef9bcfd.el9.n 30/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3870.g9ef 31/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3870.g9ef9bcfd.el 32/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3870.g9ef9bcfd.el9.noarch 33/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3870.g9ef9bcfd.e 34/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:19.0.0-3870.g9ef9bcfd.el9.noarch 35/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noarch 36/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 37/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 38/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 39/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/130 2024-05-21T17:40:44.598 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 45/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/130 2024-05-21T17:40:44.599 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 55/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 56/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 57/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 59/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 60/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/130 2024-05-21T17:40:44.600 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 77/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 78/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 79/130 2024-05-21T17:40:44.601 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 81/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : sqlite-devel-3.34.1-7.el9.x86_64 82/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 84/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 85/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 86/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 87/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 88/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 89/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 90/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 91/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 92/130 2024-05-21T17:40:44.602 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 93/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 94/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 95/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 96/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 97/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 98/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 99/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 100/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 101/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 102/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 104/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 105/130 2024-05-21T17:40:44.603 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 106/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 107/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 108/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 109/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 110/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 111/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 112/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 113/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 114/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 116/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 117/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 118/130 2024-05-21T17:40:44.604 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 119/130 2024-05-21T17:40:44.605 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 120/130 2024-05-21T17:40:44.605 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 121/130 2024-05-21T17:40:44.605 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 122/130 2024-05-21T17:40:44.605 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 123/130 2024-05-21T17:40:44.605 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 124/130 2024-05-21T17:40:44.605 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : re2-1:20211101-3.el9.x86_64 125/130 2024-05-21T17:40:44.605 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 126/130 2024-05-21T17:40:44.605 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 127/130 2024-05-21T17:40:44.605 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 128/130 2024-05-21T17:40:44.605 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 129/130 2024-05-21T17:40:46.368 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 124/130 2024-05-21T17:40:46.409 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 125/130 2024-05-21T17:40:46.432 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 130/130 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:46.433 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.434 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.435 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T17:40:46.436 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.437 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T17:40:46.438 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T17:40:46.439 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T17:40:46.440 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T17:40:46.441 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T17:40:46.442 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T17:40:46.442 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.442 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.442 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:46.442 INFO:teuthology.orchestra.run.smithi003.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T17:40:46.442 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T17:40:46.442 INFO:teuthology.orchestra.run.smithi003.stdout: sqlite-3.34.1-7.el9.x86_64 2024-05-21T17:40:46.442 INFO:teuthology.orchestra.run.smithi003.stdout: sqlite-devel-3.34.1-7.el9.x86_64 2024-05-21T17:40:46.442 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T17:40:46.443 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 126/130 2024-05-21T17:40:46.443 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T17:40:46.443 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:46.443 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-05-21T17:40:46.466 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : sqlite-devel-3.34.1-7.el9.x86_64 127/130 2024-05-21T17:40:46.488 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_ 128/130 2024-05-21T17:40:46.488 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-05-21T17:40:46.580 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-05-21T17:40:46.581 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 130/130 2024-05-21T17:40:46.761 DEBUG:teuthology.parallel:result is None 2024-05-21T17:40:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 130/130 2024-05-21T17:40:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 1/130 2024-05-21T17:40:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2/130 2024-05-21T17:40:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 3/130 2024-05-21T17:40:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 4/130 2024-05-21T17:40:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3870.g9ef9b 5/130 2024-05-21T17:40:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 6/130 2024-05-21T17:40:52.251 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 7/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 8/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 9/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 10/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 11/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 12/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_ 13/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 14/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 15/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_6 16/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:19.0.0-3870.g9ef9bcfd.el9.x86 17/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 18/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3870.g9ef9bcfd.el 19/130 2024-05-21T17:40:52.252 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:19.0.0-3870.g9ef9bcfd.el9. 20/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:19.0.0-3870.g9ef9bcfd.el9.x86_6 21/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 22/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 23/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 24/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 25/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 26/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 27/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3870.g9ef9bcfd. 28/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noa 29/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3870.g9ef9bcfd.el9.n 30/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3870.g9ef 31/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3870.g9ef9bcfd.el 32/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3870.g9ef9bcfd.el9.noarch 33/130 2024-05-21T17:40:52.253 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3870.g9ef9bcfd.e 34/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:19.0.0-3870.g9ef9bcfd.el9.noarch 35/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noarch 36/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 37/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 38/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 39/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 45/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/130 2024-05-21T17:40:52.254 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 55/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 56/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 57/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 59/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 60/130 2024-05-21T17:40:52.255 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/130 2024-05-21T17:40:52.256 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 77/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 78/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 79/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 81/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : sqlite-devel-3.34.1-7.el9.x86_64 82/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 84/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 85/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 86/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 87/130 2024-05-21T17:40:52.257 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 88/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 89/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 90/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 91/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 92/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 93/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 94/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 95/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 96/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 97/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 98/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 99/130 2024-05-21T17:40:52.258 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 100/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 101/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 102/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 104/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 105/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 106/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 107/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 108/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 109/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 110/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 111/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 112/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 113/130 2024-05-21T17:40:52.259 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 114/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 116/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 117/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 118/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 119/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 120/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 121/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 122/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 123/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 124/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-3.el9.x86_64 125/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 126/130 2024-05-21T17:40:52.260 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 127/130 2024-05-21T17:40:52.261 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 128/130 2024-05-21T17:40:52.261 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 129/130 2024-05-21T17:40:54.348 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 130/130 2024-05-21T17:40:54.348 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.350 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:19.0.0-3870.g9ef9bcfd.el9.noarch 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.351 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: lua-5.4.4-4.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-21T17:40:54.352 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-21T17:40:54.353 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-21T17:40:54.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-21T17:40:54.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-21T17:40:54.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-21T17:40:54.357 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-21T17:40:54.358 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-21T17:40:54.358 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.358 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.358 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:19.0.0-3870.g9ef9bcfd.el9.x86_64 2024-05-21T17:40:54.358 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-3.el9.x86_64 2024-05-21T17:40:54.358 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-21T17:40:54.358 INFO:teuthology.orchestra.run.smithi203.stdout: sqlite-3.34.1-7.el9.x86_64 2024-05-21T17:40:54.358 INFO:teuthology.orchestra.run.smithi203.stdout: sqlite-devel-3.34.1-7.el9.x86_64 2024-05-21T17:40:54.358 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-21T17:40:54.359 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-21T17:40:54.359 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:54.359 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-05-21T17:40:54.713 DEBUG:teuthology.parallel:result is None 2024-05-21T17:40:54.714 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 2024-05-21T17:40:54.939 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T17:40:54.966 INFO:teuthology.orchestra.run.smithi003.stdout:19.0.0-3870.g9ef9bcfd.el9 2024-05-21T17:40:54.967 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3870.g9ef9bcfd.el9 2024-05-21T17:40:54.967 INFO:teuthology.task.install:The correct ceph version 19.0.0-3870.g9ef9bcfd is installed. 2024-05-21T17:40:54.969 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 2024-05-21T17:40:55.109 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-21T17:40:55.132 INFO:teuthology.orchestra.run.smithi203.stdout:19.0.0-3870.g9ef9bcfd.el9 2024-05-21T17:40:55.133 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3870.g9ef9bcfd.el9 2024-05-21T17:40:55.133 INFO:teuthology.task.install:The correct ceph version 19.0.0-3870.g9ef9bcfd is installed. 2024-05-21T17:40:55.134 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-21T17:40:55.135 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:55.135 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T17:40:55.168 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:40:55.168 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-21T17:40:55.201 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-21T17:40:55.201 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:55.201 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T17:40:55.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T17:40:55.308 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:40:55.309 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-05-21T17:40:55.333 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-21T17:40:55.398 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-21T17:40:55.398 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:55.398 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T17:40:55.427 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T17:40:55.497 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:40:55.497 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-21T17:40:55.521 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-21T17:40:55.586 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-21T17:40:55.587 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:55.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T17:40:55.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T17:40:55.685 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:40:55.685 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-05-21T17:40:55.709 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-21T17:40:55.774 INFO:teuthology.run_tasks:Running task ceph... 2024-05-21T17:40:55.849 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-21T17:40:55.850 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /var/log/ceph 2024-05-21T17:40:55.852 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /var/log/ceph 2024-05-21T17:40:55.879 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-21T17:40:55.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-21T17:40:55.923 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-21T17:40:55.949 INFO:tasks.ceph:Creating extra log directories... 2024-05-21T17:40:55.949 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-21T17:40:55.993 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-21T17:40:56.397 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-21T17:40:56.397 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-21T17:40:56.397 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-21T17:40:56.397 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:56.397 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-21T17:40:56.419 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-21T17:40:56.542 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-21T17:40:56.571 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:40:56.571 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-21T17:40:56.588 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-21T17:40:56.703 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-21T17:40:56.736 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-21T17:40:56.736 INFO:tasks.ceph:config {'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 0}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_', 'replacing it with standby', 'No standby daemons available', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-21T17:40:56.736 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528', 'branch': 'wip-yuri7-testing-2024-05-20-1227', 'description': 'rados/mgr/{clusters/{2-node-mgr} debug/mgr distro/{centos_latest} mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/{1-install 2-ceph 3-mgrmodules 4-units/workunits}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7717528', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 2, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2024-05-20-1227'}, 'ceph': {'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 0}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri7-testing-2024-05-20-1227', 'sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'], ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1']], 'seed': 4736, 'sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri7-testing-2024-05-20-1227', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_9ef9bcfdb32ec4999d7ca025815290e8395c6ae7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7', 'targets': {'smithi003.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVpevLpr+7TbmJfvsgGCXP8LXqBJdnA+8DctMAGvYIiAvkNib/r6aPZkRp1QMdGmeAzBdE4B5EcqTsHLd04hpM=', 'smithi203.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwQaEx4f5aPRqCDCqzJi44+In4ryYKNFY4kyLj+LNZ/Tlhk3ZPZrHqWcIJYL7k+y/YJ06lUJYpAYXBP1LHNPMA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7'}}, {'ceph': {'log-ignorelist': ['overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_', 'replacing it with standby', 'No standby daemons available', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'wait-for-scrub': False, 'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 0}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'sha1': '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'workunit': {'clients': {'client.0': ['mgr/test_localpool.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b9e3da87b1ec7920937cb4902b7e3fd9eda86b4a', 'timestamp': '2024-05-21_01:07:55', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841'} 2024-05-21T17:40:56.737 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-21T17:40:56.740 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-21T17:40:56.787 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m0777 -- /var/run/ceph 2024-05-21T17:40:56.800 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/run/ceph 2024-05-21T17:40:56.852 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:56.853 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T17:40:56.885 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T17:40:56.886 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-05-21T17:40:56.944 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T17:40:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T17:40:56.945 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-21T17:40:56.945 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T17:40:56.945 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T17:40:56.945 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-21 17:40:39.729309380 +0000 2024-05-21T17:40:56.945 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-21 17:38:37.962276538 +0000 2024-05-21T17:40:56.945 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-21 17:38:37.962276538 +0000 2024-05-21T17:40:56.945 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-21 17:38:37.962276538 +0000 2024-05-21T17:40:56.945 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T17:40:57.012 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-21T17:40:57.013 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-21T17:40:57.013 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000269246 s, 1.9 MB/s 2024-05-21T17:40:57.014 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T17:40:57.074 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-05-21T17:40:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T17:40:57.133 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T17:40:57.133 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-21T17:40:57.133 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T17:40:57.134 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T17:40:57.134 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-21 17:40:39.730309367 +0000 2024-05-21T17:40:57.134 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-21 17:38:38.312272137 +0000 2024-05-21T17:40:57.134 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-21 17:38:38.312272137 +0000 2024-05-21T17:40:57.134 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-21 17:38:38.312272137 +0000 2024-05-21T17:40:57.134 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T17:40:57.201 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-21T17:40:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-21T17:40:57.202 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000257592 s, 2.0 MB/s 2024-05-21T17:40:57.203 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T17:40:57.262 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-05-21T17:40:57.319 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T17:40:57.319 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T17:40:57.320 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-21T17:40:57.320 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T17:40:57.320 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T17:40:57.320 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-21 17:40:39.730309367 +0000 2024-05-21T17:40:57.320 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-21 17:38:38.645267950 +0000 2024-05-21T17:40:57.320 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-21 17:38:38.645267950 +0000 2024-05-21T17:40:57.320 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-21 17:38:38.645267950 +0000 2024-05-21T17:40:57.320 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T17:40:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-21T17:40:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-21T17:40:57.386 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000332698 s, 1.5 MB/s 2024-05-21T17:40:57.388 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T17:40:57.446 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-05-21T17:40:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T17:40:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T17:40:57.504 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-21T17:40:57.504 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T17:40:57.504 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T17:40:57.504 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-21 17:40:39.730309367 +0000 2024-05-21T17:40:57.504 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-21 17:38:38.980263737 +0000 2024-05-21T17:40:57.504 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-21 17:38:38.980263737 +0000 2024-05-21T17:40:57.504 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-21 17:38:38.980263737 +0000 2024-05-21T17:40:57.505 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T17:40:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-21T17:40:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-21T17:40:57.569 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000232957 s, 2.2 MB/s 2024-05-21T17:40:57.571 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T17:40:57.628 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-05-21T17:40:57.628 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:40:57.628 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-05-21T17:40:57.645 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-21T17:40:57.645 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-05-21T17:40:57.702 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-21T17:40:57.702 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T17:40:57.702 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-21T17:40:57.702 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T17:40:57.702 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T17:40:57.702 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-21 17:40:47.031332004 +0000 2024-05-21T17:40:57.702 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-21 17:38:34.786938442 +0000 2024-05-21T17:40:57.702 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-21 17:38:34.786938442 +0000 2024-05-21T17:40:57.702 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-21 17:38:34.786938442 +0000 2024-05-21T17:40:57.703 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-21T17:40:57.767 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-21T17:40:57.767 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-21T17:40:57.767 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000190057 s, 2.7 MB/s 2024-05-21T17:40:57.768 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-21T17:40:57.826 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-05-21T17:40:57.881 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-21T17:40:57.881 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T17:40:57.881 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-21T17:40:57.882 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T17:40:57.882 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T17:40:57.882 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-21 17:40:47.032331980 +0000 2024-05-21T17:40:57.882 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-21 17:38:35.127929902 +0000 2024-05-21T17:40:57.882 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-21 17:38:35.127929902 +0000 2024-05-21T17:40:57.882 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-21 17:38:35.127929902 +0000 2024-05-21T17:40:57.882 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-21T17:40:57.946 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-21T17:40:57.946 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-21T17:40:57.946 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000211974 s, 2.4 MB/s 2024-05-21T17:40:57.947 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-21T17:40:58.004 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-05-21T17:40:58.060 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-21T17:40:58.060 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T17:40:58.060 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-21T17:40:58.060 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T17:40:58.060 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T17:40:58.060 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-21 17:40:47.032331980 +0000 2024-05-21T17:40:58.061 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-21 17:38:35.487920885 +0000 2024-05-21T17:40:58.061 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-21 17:38:35.487920885 +0000 2024-05-21T17:40:58.061 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-21 17:38:35.487920885 +0000 2024-05-21T17:40:58.061 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-21T17:40:58.124 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-21T17:40:58.124 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-21T17:40:58.124 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000188482 s, 2.7 MB/s 2024-05-21T17:40:58.125 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-21T17:40:58.182 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-05-21T17:40:58.238 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-21T17:40:58.238 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-21T17:40:58.238 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-21T17:40:58.238 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-21T17:40:58.238 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-05-21T17:40:58.239 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-05-21 17:40:47.032331980 +0000 2024-05-21T17:40:58.239 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-05-21 17:38:35.817912620 +0000 2024-05-21T17:40:58.239 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-05-21 17:38:35.817912620 +0000 2024-05-21T17:40:58.239 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-05-21 17:38:35.817912620 +0000 2024-05-21T17:40:58.239 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-21T17:40:58.301 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-05-21T17:40:58.301 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-05-21T17:40:58.301 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000238656 s, 2.1 MB/s 2024-05-21T17:40:58.302 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-21T17:40:58.359 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-05-21T17:40:58.359 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-05-21T17:40:58.359 INFO:tasks.ceph:Generating config... 2024-05-21T17:40:58.360 INFO:tasks.ceph:[client] debug client = 20 2024-05-21T17:40:58.360 INFO:tasks.ceph:[client] debug mgrc = 20 2024-05-21T17:40:58.360 INFO:tasks.ceph:[client] debug ms = 1 2024-05-21T17:40:58.361 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-05-21T17:40:58.361 INFO:tasks.ceph:[mds] debug mgrc = 20 2024-05-21T17:40:58.361 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-21T17:40:58.361 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-21T17:40:58.361 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-21T17:40:58.361 INFO:tasks.ceph:[mgr] mgr ttl cache expire seconds = 0 2024-05-21T17:40:58.361 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-21T17:40:58.361 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-21T17:40:58.361 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-21T17:40:58.361 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-21T17:40:58.361 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-05-21T17:40:58.361 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-05-21T17:40:58.361 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-21T17:40:58.361 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-21T17:40:58.361 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-21T17:40:58.362 INFO:tasks.ceph:[osd] debug mgrc = 20 2024-05-21T17:40:58.362 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-21T17:40:58.362 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-21T17:40:58.362 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-21T17:40:58.362 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-21T17:40:58.362 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-21T17:40:58.362 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-21T17:40:58.362 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-21T17:40:58.362 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-21T17:40:58.362 INFO:tasks.ceph:Setting up mon.a... 2024-05-21T17:40:58.363 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-21T17:40:58.422 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.keyring 2024-05-21T17:40:58.425 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-21T17:40:58.474 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-21T17:40:58.501 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.3'), ('mon.c', '[v2:172.21.15.3:3301,v1:172.21.15.3:6790]'), ('mon.b', '172.21.15.203')] 2024-05-21T17:40:58.501 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.3,[v2:172.21.15.3:3301,v1:172.21.15.3:6790],172.21.15.203', 'mon election default strategy': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'mgr ttl cache expire seconds': 0}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds': {'debug mgrc': 20}} 2024-05-21T17:40:58.502 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:58.502 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-21T17:40:58.557 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.3 --addv c '[v2:172.21.15.3:3301,v1:172.21.15.3:6790]' --add b 172.21.15.203 --print /home/ubuntu/cephtest/ceph.monmap 2024-05-21T17:40:58.647 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-21T17:40:58.647 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: generated fsid e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442 2024-05-21T17:40:58.647 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = quincy 2024-05-21T17:40:58.647 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 0 2024-05-21T17:40:58.647 INFO:teuthology.orchestra.run.smithi003.stdout:fsid e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442 2024-05-21T17:40:58.647 INFO:teuthology.orchestra.run.smithi003.stdout:last_changed 2024-05-21T17:40:58.646640+0000 2024-05-21T17:40:58.647 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-05-21T17:40:58.646640+0000 2024-05-21T17:40:58.647 INFO:teuthology.orchestra.run.smithi003.stdout:min_mon_release 17 (quincy) 2024-05-21T17:40:58.647 INFO:teuthology.orchestra.run.smithi003.stdout:election_strategy: 1 2024-05-21T17:40:58.648 INFO:teuthology.orchestra.run.smithi003.stdout:0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-05-21T17:40:58.648 INFO:teuthology.orchestra.run.smithi003.stdout:1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.b 2024-05-21T17:40:58.648 INFO:teuthology.orchestra.run.smithi003.stdout:2: [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] mon.c 2024-05-21T17:40:58.648 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-21T17:40:58.649 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-21T17:40:58.665 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442... 2024-05-21T17:40:58.665 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-21T17:40:58.707 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-21T17:40:58.749 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-05-21T17:40:58.749 INFO:teuthology.orchestra.run.smithi003.stdout: chdir = "" 2024-05-21T17:40:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-21T17:40:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: auth supported = cephx 2024-05-21T17:40:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: filestore xattr use omap = true 2024-05-21T17:40:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: mon clock drift allowed = 1.000 2024-05-21T17:40:58.750 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: osd crush chooseleaf type = 0 2024-05-21T17:40:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: auth debug = true 2024-05-21T17:40:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on old message = true 2024-05-21T17:40:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: ms die on bug = true 2024-05-21T17:40:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-21T17:40:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: mon pg warn max object skew = 0 2024-05-21T17:40:58.751 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: # disable pg_autoscaler by default for new pools 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default size = 2 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow primary affinity = true 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd allow pg remap = true 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on legacy crush tunables = false 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on crush straw calc version zero = false 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on no sortbitwise = false 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on osd down out interval zero = false 2024-05-21T17:40:58.752 INFO:teuthology.orchestra.run.smithi003.stdout: mon warn on too few osds = false 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: mon_allow_pool_size_one = true 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: osd default data pool replay window = 5 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: mon allow pool delete = true 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: mon cluster log file level = debug 2024-05-21T17:40:58.753 INFO:teuthology.orchestra.run.smithi003.stdout: debug asserts on shutdown = true 2024-05-21T17:40:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: mon health detail to clog = false 2024-05-21T17:40:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: mon host = "172.21.15.3,[v2:172.21.15.3:3301,v1:172.21.15.3:6790],172.21.15.203" 2024-05-21T17:40:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: mon election default strategy = 1 2024-05-21T17:40:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442 2024-05-21T17:40:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.754 INFO:teuthology.orchestra.run.smithi003.stdout:[osd] 2024-05-21T17:40:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: osd journal size = 100 2024-05-21T17:40:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.754 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub load threshold = 5.0 2024-05-21T17:40:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: osd scrub max interval = 600 2024-05-21T17:40:58.755 INFO:teuthology.orchestra.run.smithi003.stdout: osd mclock profile = high_recovery_ops 2024-05-21T17:40:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: osd recover clone overlap = true 2024-05-21T17:40:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: osd recovery max chunk = 1048576 2024-05-21T17:40:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.756 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug shutdown = true 2024-05-21T17:40:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug op order = true 2024-05-21T17:40:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug verify stray on activate = true 2024-05-21T17:40:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug trim objects = true 2024-05-21T17:40:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: osd open classes on start = true 2024-05-21T17:40:58.757 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug pg log writeout = true 2024-05-21T17:40:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: osd deep scrub update digest min age = 30 2024-05-21T17:40:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.758 INFO:teuthology.orchestra.run.smithi003.stdout: osd map max advance = 10 2024-05-21T17:40:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: journal zero on create = true 2024-05-21T17:40:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: filestore ondisk finisher threads = 3 2024-05-21T17:40:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: filestore apply finisher threads = 3 2024-05-21T17:40:58.759 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: bdev debug aio = true 2024-05-21T17:40:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: osd debug misdirected ops = true 2024-05-21T17:40:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore block size = 96636764160 2024-05-21T17:40:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore compression algorithm = lz4 2024-05-21T17:40:58.760 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore compression mode = aggressive 2024-05-21T17:40:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: bluestore fsck on mount = True 2024-05-21T17:40:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluefs = 20 2024-05-21T17:40:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: debug bluestore = 20 2024-05-21T17:40:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgrc = 20 2024-05-21T17:40:58.761 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: debug osd = 20 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: debug rocksdb = 10 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd backfillfull_ratio = 0.85 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd full ratio = 0.9 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd nearfull ratio = 0.8 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: osd failsafe full ratio = 0.95 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: osd objectstore = bluestore 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr] 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgr = 20 2024-05-21T17:40:58.762 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: mgr/telemetry/nag = false 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: mgr ttl cache expire seconds = 0 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout:[mon] 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: debug mon = 20 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: debug paxos = 20 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: debug auth = 20 2024-05-21T17:40:58.763 INFO:teuthology.orchestra.run.smithi003.stdout: mon data avail warn = 5 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon mgr mkfs grace = 240 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min pgs per osd = 4 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd reporter subtree level = osd 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon osd prime pg temp = true 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon reweight min bytes per osd = 10 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: auth mon ticket ttl = 660 # 11m 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: auth service ticket ttl = 240 # 4m 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: # don't complain about insecure global_id in the test suite 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-21T17:40:58.764 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: # 1m isn't quite enough 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: mon_down_mkfs_grace = 2m 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: mon_warn_on_filestore_osds = false 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout:[client] 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: rgw cache enabled = true 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable ops log = true 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: rgw enable usage log = true 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: debug client = 20 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgrc = 20 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout: debug ms = 1 2024-05-21T17:40:58.765 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.a] 2024-05-21T17:40:58.766 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.c] 2024-05-21T17:40:58.766 INFO:teuthology.orchestra.run.smithi003.stdout:[mon.b] 2024-05-21T17:40:58.766 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-05-21T17:40:58.766 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-05-21T17:40:58.766 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-05-21T17:40:58.766 INFO:teuthology.orchestra.run.smithi003.stdout:[mds] 2024-05-21T17:40:58.766 INFO:teuthology.orchestra.run.smithi003.stdout: debug mgrc = 20 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-05-21T17:40:58.768 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-05-21T17:40:58.769 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-05-21T17:40:58.770 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.3,[v2:172.21.15.3:3301,v1:172.21.15.3:6790],172.21.15.203" 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: mon election default strategy = 1 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.771 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-05-21T17:40:58.772 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore compression algorithm = lz4 2024-05-21T17:40:58.773 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore compression mode = aggressive 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 20 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 20 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgrc = 20 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 10 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-05-21T17:40:58.774 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: mgr ttl cache expire seconds = 0 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-05-21T17:40:58.775 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.776 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-05-21T17:40:58.777 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgrc = 20 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-05-21T17:40:58.778 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgrc = 20 2024-05-21T17:40:58.779 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-21T17:40:58.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-21T17:40:58.864 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-21T17:40:58.864 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:58.864 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-21T17:40:58.880 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:58.880 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-21T17:40:58.937 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi003.front.sepia.ceph.com 2024-05-21T17:40:58.937 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:58.937 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-21T17:40:58.937 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-21T17:40:59.012 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:40:59.012 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-21T17:40:59.069 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi203.front.sepia.ceph.com 2024-05-21T17:40:59.069 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:40:59.069 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-21T17:40:59.069 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-21T17:40:59.104 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:40:59.104 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-21T17:40:59.161 INFO:tasks.ceph:Setting up mon nodes... 2024-05-21T17:40:59.161 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-21T17:40:59.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-21T17:40:59.214 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-21T17:40:59.217 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-21T17:40:59.299 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-21T17:40:59.302 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mgr/ceph-z && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.z /var/lib/ceph/mgr/ceph-z/keyring 2024-05-21T17:40:59.352 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mgr/ceph-z/keyring 2024-05-21T17:40:59.355 INFO:tasks.ceph:Setting up mds nodes... 2024-05-21T17:40:59.355 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-21T17:40:59.409 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-21T17:40:59.410 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-21T17:40:59.436 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-21T17:40:59.528 INFO:teuthology.orchestra.run.smithi003.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-21T17:40:59.531 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-21T17:40:59.556 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-21T17:40:59.607 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-21T17:40:59.610 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-21T17:40:59.635 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-21T17:40:59.635 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-21T17:40:59.678 INFO:teuthology.orchestra.run.smithi003.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-21T17:40:59.691 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-21T17:40:59.731 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-21T17:40:59.744 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-21T17:40:59.745 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi003.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-05-21T17:40:59.745 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-21T17:40:59.770 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-05-21T17:40:59.770 INFO:tasks.ceph:role: osd.0 2024-05-21T17:40:59.771 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com 2024-05-21T17:40:59.771 DEBUG:teuthology.orchestra.run.smithi003:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-21T17:40:59.843 INFO:teuthology.orchestra.run.smithi003.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-21T17:40:59.843 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-21T17:40:59.844 INFO:teuthology.orchestra.run.smithi003.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-21T17:40:59.844 INFO:teuthology.orchestra.run.smithi003.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-21T17:40:59.844 INFO:teuthology.orchestra.run.smithi003.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-21T17:40:59.844 INFO:teuthology.orchestra.run.smithi003.stdout: = sunit=0 swidth=0 blks 2024-05-21T17:40:59.844 INFO:teuthology.orchestra.run.smithi003.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-21T17:40:59.844 INFO:teuthology.orchestra.run.smithi003.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-21T17:40:59.844 INFO:teuthology.orchestra.run.smithi003.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-21T17:40:59.844 INFO:teuthology.orchestra.run.smithi003.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-21T17:41:00.342 INFO:teuthology.orchestra.run.smithi003.stdout:Discarding blocks...Done. 2024-05-21T17:41:00.344 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi003.front.sepia.ceph.com -o noatime 2024-05-21T17:41:00.344 DEBUG:teuthology.orchestra.run.smithi003:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-21T17:41:00.475 DEBUG:teuthology.orchestra.run.smithi003:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-21T17:41:00.510 DEBUG:teuthology.orchestra.run.smithi003:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-21T17:41:00.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:00.619+0000 7f2fe6210640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-21T17:41:00.622 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:00.622+0000 7f2fe6210640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-21T17:41:00.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:00.622+0000 7f2fe6210640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-21T17:41:00.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:00.622+0000 7f2fe6210640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-21T17:41:00.623 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:00.622+0000 7f2fe6210640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-21T17:41:00.627 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:00.626+0000 7f2fe6210640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-21T17:41:03.491 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-21T17:41:03.524 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-21T17:41:03.554 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-05-21T17:41:03.554 INFO:tasks.ceph:role: osd.1 2024-05-21T17:41:03.554 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com 2024-05-21T17:41:03.554 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-21T17:41:03.630 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-21T17:41:03.630 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-21T17:41:03.630 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-21T17:41:03.630 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-21T17:41:03.630 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-21T17:41:03.630 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-05-21T17:41:03.630 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-21T17:41:03.631 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-21T17:41:03.631 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-21T17:41:03.631 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-21T17:41:04.126 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-05-21T17:41:04.130 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-05-21T17:41:04.130 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-05-21T17:41:04.247 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-21T17:41:04.281 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-21T17:41:04.349 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-05-21T17:41:04.349 INFO:tasks.ceph:role: osd.2 2024-05-21T17:41:04.349 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com 2024-05-21T17:41:04.349 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-21T17:41:04.424 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-21T17:41:04.424 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-21T17:41:04.425 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-21T17:41:04.425 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-21T17:41:04.425 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-21T17:41:04.425 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-05-21T17:41:04.425 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-21T17:41:04.425 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-21T17:41:04.425 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-21T17:41:04.425 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-21T17:41:04.884 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-05-21T17:41:04.886 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-05-21T17:41:04.887 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2024-05-21T17:41:04.977 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-21T17:41:05.005 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-21T17:41:05.072 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-05-21T17:41:05.072 INFO:tasks.ceph:role: osd.3 2024-05-21T17:41:05.072 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com 2024-05-21T17:41:05.072 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-21T17:41:05.145 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-21T17:41:05.145 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-21T17:41:05.146 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-21T17:41:05.146 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-21T17:41:05.146 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-21T17:41:05.146 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-05-21T17:41:05.146 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-21T17:41:05.146 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-21T17:41:05.146 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-21T17:41:05.146 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-21T17:41:05.624 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-05-21T17:41:05.626 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-05-21T17:41:05.626 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2024-05-21T17:41:05.764 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-21T17:41:05.796 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-21T17:41:05.918 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:05.916+0000 7f4ddaf0a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-21T17:41:05.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:05.919+0000 7f4ddaf0a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-21T17:41:05.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:05.919+0000 7f4ddaf0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-21T17:41:05.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:05.919+0000 7f4ddaf0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-21T17:41:05.921 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:05.919+0000 7f4ddaf0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-21T17:41:05.923 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:05.921+0000 7f4ddaf0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-21T17:41:08.782 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-21T17:41:08.809 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-21T17:41:08.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:08.897+0000 7f6e5ba1f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-21T17:41:08.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:08.897+0000 7f6e5ba1f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-21T17:41:08.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:08.897+0000 7f6e5ba1f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-21T17:41:08.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:08.897+0000 7f6e5ba1f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-21T17:41:08.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:08.897+0000 7f6e5ba1f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-21T17:41:08.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:08.898+0000 7f6e5ba1f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-21T17:41:11.744 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-21T17:41:11.774 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-21T17:41:11.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:11.863+0000 7f3036fa5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-21T17:41:11.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:11.863+0000 7f3036fa5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-21T17:41:11.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:11.863+0000 7f3036fa5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-21T17:41:11.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:11.863+0000 7f3036fa5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-21T17:41:11.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:11.863+0000 7f3036fa5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-21T17:41:11.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:11.864+0000 7f3036fa5640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-21T17:41:14.725 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-21T17:41:14.753 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-21T17:41:14.754 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:41:14.754 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-21T17:41:14.786 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:41:14.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-21T17:41:14.857 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:41:14.857 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-21T17:41:14.927 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:41:14.927 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-21T17:41:14.995 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:41:14.995 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-21T17:41:15.025 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:41:15.025 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mgr/ceph-z/keyring of=/dev/stdout 2024-05-21T17:41:15.096 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:41:15.096 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-21T17:41:15.164 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:41:15.164 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-21T17:41:15.231 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:41:15.231 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-21T17:41:15.299 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:41:15.299 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-21T17:41:15.365 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:41:15.365 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-21T17:41:15.385 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:41:15.385 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-21T17:41:15.422 INFO:tasks.ceph:Adding keys to all mons... 2024-05-21T17:41:15.423 DEBUG:teuthology.orchestra.run.smithi003:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-21T17:41:15.428 DEBUG:teuthology.orchestra.run.smithi203:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-21T17:41:15.466 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.x] 2024-05-21T17:41:15.466 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCr3ExmQpnLDBAAPYOwREmfBk9w0UdrCiA7RQ== 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a] 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCr3ExmdOdOGBAAi2fTO9KEB6I5tQTcDXpbSQ== 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c] 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCr3ExmwLR+HxAAmyXoPcDAhedZ7rIfpF86zg== 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.0] 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCs3Exm6Vj1JBAAjazjGtJUH6A54sr9qek8Bw== 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.y] 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCr3Exmpa7XERAAUXI7JoOAXaRJ6DRdSIAPqg== 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:[mgr.z] 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCr3Exm31cAFRAAuLK7hJuQ69zf8jilSJ30DA== 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b] 2024-05-21T17:41:15.467 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCr3Exmitg3JBAAvRIo7EJ1tmXDau3cBoAotA== 2024-05-21T17:41:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.1] 2024-05-21T17:41:15.468 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCx3ExmQNi7NhAARFvlfNUkLtMN3XdgqG0j3Q== 2024-05-21T17:41:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.2] 2024-05-21T17:41:15.468 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQC03Exm/SqINRAAIFshknJZbFb8vCR0KtPmJQ== 2024-05-21T17:41:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:[osd.3] 2024-05-21T17:41:15.468 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQC33Exm5leGMxAAVg8TKuLESUG3TNzT1dOnRA== 2024-05-21T17:41:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-05-21T17:41:15.468 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCr3Exm97NqKBAASumBWpLJbxmmgJLPhFwpiw== 2024-05-21T17:41:15.468 INFO:teuthology.orchestra.run.smithi003.stdout:[client.1] 2024-05-21T17:41:15.468 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQCr3ExmmHGhKxAA/OOsm/Ckd0KaieKi0oDMgA== 2024-05-21T17:41:15.487 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.x] 2024-05-21T17:41:15.487 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCr3ExmQpnLDBAAPYOwREmfBk9w0UdrCiA7RQ== 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCr3ExmdOdOGBAAi2fTO9KEB6I5tQTcDXpbSQ== 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCr3ExmwLR+HxAAmyXoPcDAhedZ7rIfpF86zg== 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.0] 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCs3Exm6Vj1JBAAjazjGtJUH6A54sr9qek8Bw== 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.y] 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCr3Exmpa7XERAAUXI7JoOAXaRJ6DRdSIAPqg== 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.z] 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCr3Exm31cAFRAAuLK7hJuQ69zf8jilSJ30DA== 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCr3Exmitg3JBAAvRIo7EJ1tmXDau3cBoAotA== 2024-05-21T17:41:15.488 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.1] 2024-05-21T17:41:15.489 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCx3ExmQNi7NhAARFvlfNUkLtMN3XdgqG0j3Q== 2024-05-21T17:41:15.489 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.2] 2024-05-21T17:41:15.489 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQC03Exm/SqINRAAIFshknJZbFb8vCR0KtPmJQ== 2024-05-21T17:41:15.489 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.3] 2024-05-21T17:41:15.489 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQC33Exm5leGMxAAVg8TKuLESUG3TNzT1dOnRA== 2024-05-21T17:41:15.489 INFO:teuthology.orchestra.run.smithi203.stdout:[client.0] 2024-05-21T17:41:15.489 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCr3Exm97NqKBAASumBWpLJbxmmgJLPhFwpiw== 2024-05-21T17:41:15.489 INFO:teuthology.orchestra.run.smithi203.stdout:[client.1] 2024-05-21T17:41:15.489 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCr3ExmmHGhKxAA/OOsm/Ckd0KaieKi0oDMgA== 2024-05-21T17:41:15.490 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-21T17:41:15.512 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-21T17:41:15.581 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-21T17:41:15.584 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-21T17:41:15.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-21T17:41:15.642 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-21T17:41:15.696 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-21T17:41:15.698 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-21T17:41:15.749 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-21T17:41:15.752 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-21T17:41:15.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.z --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-21T17:41:15.805 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.z --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-21T17:41:15.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-21T17:41:15.858 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-21T17:41:15.912 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-21T17:41:15.915 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-21T17:41:15.965 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-21T17:41:15.968 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-21T17:41:16.018 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-21T17:41:16.021 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-21T17:41:16.071 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-21T17:41:16.073 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-21T17:41:16.126 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-21T17:41:16.129 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-21T17:41:16.182 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-21T17:41:16.182 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-21T17:41:16.208 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-21T17:41:16.305 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-21T17:41:16.331 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-21T17:41:16.396 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-21T17:41:16.497 DEBUG:teuthology.orchestra.run.smithi003:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-21T17:41:16.523 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-21T17:41:16.550 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-21T17:41:16.652 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-21T17:41:16.677 DEBUG:teuthology.orchestra.run.smithi003:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-21T17:41:16.679 DEBUG:teuthology.orchestra.run.smithi203:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-21T17:41:16.732 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-21T17:41:16.732 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-21T17:41:16.732 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-21T17:41:16.735 INFO:tasks.ceph.mon.a:Started 2024-05-21T17:41:16.735 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-21T17:41:16.735 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-21T17:41:16.737 INFO:tasks.ceph.mon.c:Started 2024-05-21T17:41:16.737 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-21T17:41:16.737 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-21T17:41:16.774 INFO:tasks.ceph.mon.b:Started 2024-05-21T17:41:16.774 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-21T17:41:16.774 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-21T17:41:16.774 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-21T17:41:16.777 INFO:tasks.ceph.mgr.x:Started 2024-05-21T17:41:16.777 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-21T17:41:16.777 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-21T17:41:16.779 INFO:tasks.ceph.mgr.y:Started 2024-05-21T17:41:16.779 INFO:tasks.ceph.mgr.z:Restarting daemon 2024-05-21T17:41:16.779 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i z 2024-05-21T17:41:16.781 INFO:tasks.ceph.mgr.z:Started 2024-05-21T17:41:16.781 INFO:tasks.ceph:Setting crush tunables to default 2024-05-21T17:41:16.781 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd crush tunables default 2024-05-21T17:41:16.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.885+0000 7f346adc8640 1 Processor -- start 2024-05-21T17:41:16.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.885+0000 7f346adc8640 1 -- start start 2024-05-21T17:41:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f346adc8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 0x7f34640a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f346adc8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3464073e60 0x7f34640a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f346adc8640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34640a8380 con 0x7f3464146240 2024-05-21T17:41:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f346adc8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f346406e2a0 con 0x7f3464073e60 2024-05-21T17:41:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f346adc8640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f346406e4d0 con 0x7f34640a8a80 2024-05-21T17:41:16.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3468b64640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 0x7f34640a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3469365640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f3464146240 0x7f34641466f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33170/0 (socket says 172.21.15.3:33170) 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3469365640 1 -- 172.21.15.3:0/1952167384 learned_addr learned my addr 172.21.15.3:0/1952167384 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3468b64640 1 -- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 msgr2=0x7f34640a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3468b64640 1 -- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 msgr2=0x7f34640a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3468b64640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 0x7f34640a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3468b64640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 0x7f34640a1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3463fff640 1 --2- 172.21.15.3:0/1952167384 >> v2:172.21.15.203:3300/0 conn(0x7f3464073e60 0x7f34640a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3463fff640 1 -- 172.21.15.3:0/1952167384 >> v2:172.21.15.203:3300/0 conn(0x7f3464073e60 msgr2=0x7f34640a7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3463fff640 1 -- 172.21.15.3:0/1952167384 >> v2:172.21.15.203:3300/0 conn(0x7f3464073e60 msgr2=0x7f34640a7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.886+0000 7f3463fff640 1 --2- 172.21.15.3:0/1952167384 >> v2:172.21.15.203:3300/0 conn(0x7f3464073e60 0x7f34640a7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T17:41:16.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:16.887+0000 7f3463fff640 1 --2- 172.21.15.3:0/1952167384 >> v2:172.21.15.203:3300/0 conn(0x7f3464073e60 0x7f34640a7e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T17:41:17.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.047+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 212004978 0 0) 0x7f34640a8380 con 0x7f3464146240 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.047+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f344c005730 con 0x7f3464146240 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.047+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1817672065 0 0) 0x7f344c005730 con 0x7f3464146240 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.047+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 msgr2=0x7f34640a1350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.047+0000 7f34637fe640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 0x7f34640a1350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.047+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 >> v2:172.21.15.203:3300/0 conn(0x7f3464073e60 msgr2=0x7f34640a7e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.047+0000 7f34637fe640 1 --2- 172.21.15.3:0/1952167384 >> v2:172.21.15.203:3300/0 conn(0x7f3464073e60 0x7f34640a7e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.047+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34640a8e60 con 0x7f3464146240 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.047+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3544809896 0 0) 0x7f345c0016f0 con 0x7f3464146240 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 >> v1:172.21.15.3:6789/0 conn(0x7f3464146240 legacy=0x7f34641466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f34637fe640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f344c005c60 0x7f344c006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f34637fe640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 0x7f344c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f34637fe640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c006fa0 0x7f344c007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f346406e4d0 con 0x7f344c005c60 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f344c00de30 con 0x7f344c006fa0 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f344c00e030 con 0x7f34640a8a80 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f3463fff640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 0x7f344c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f3469365640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c006fa0 0x7f344c007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f3468b64640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f344c005c60 0x7f344c006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f3468b64640 1 -- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 msgr2=0x7f344c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f3468b64640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f34640a8a80 0x7f344c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f3468b64640 1 -- 172.21.15.3:0/1952167384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c006fa0 msgr2=0x7f344c007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f3468b64640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c006fa0 0x7f344c007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.048+0000 7f3468b64640 1 -- 172.21.15.3:0/1952167384 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344c005730 con 0x7f344c005c60 2024-05-21T17:41:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.049+0000 7f3468b64640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f344c005c60 0x7f344c006080 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3454006fd0 tx=0x7f344c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6f0efcf6c991d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.049+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454036070 con 0x7f344c005c60 2024-05-21T17:41:17.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.050+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3454030740 con 0x7f344c005c60 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.050+0000 7f34637fe640 1 -- 172.21.15.3:0/1952167384 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345400f9f0 con 0x7f344c005c60 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.050+0000 7f346adc8640 1 -- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f344c005c60 msgr2=0x7f344c006080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.050+0000 7f346adc8640 1 --2- 172.21.15.3:0/1952167384 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f344c005c60 0x7f344c006080 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3454006fd0 tx=0x7f344c007a40 comp rx=0 tx=0).stop 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.050+0000 7f3469365640 1 -- 172.21.15.3:0/1952167384 reap_dead start 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.050+0000 7f346adc8640 1 -- 172.21.15.3:0/1952167384 shutdown_connections 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.050+0000 7f346adc8640 1 -- 172.21.15.3:0/1952167384 >> 172.21.15.3:0/1952167384 conn(0x7f346406ccf0 msgr2=0x7f34640a6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.050+0000 7f346adc8640 1 -- 172.21.15.3:0/1952167384 shutdown_connections 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.050+0000 7f346adc8640 1 -- 172.21.15.3:0/1952167384 wait complete. 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.051+0000 7f346adc8640 1 Processor -- start 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.051+0000 7f346adc8640 1 -- start start 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f346adc8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c005c60 0x7f3464071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f346adc8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34641ceb20 0x7f3464070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f346adc8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464072ec0 0x7f34641da920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f346adc8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f34640a59d0 con 0x7f34641ceb20 2024-05-21T17:41:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f346adc8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f34640a5850 con 0x7f344c005c60 2024-05-21T17:41:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f346adc8640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f3464146e00 con 0x7f3464072ec0 2024-05-21T17:41:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f3463fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34641ceb20 0x7f3464070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f3468b64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c005c60 0x7f3464071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f3468b64640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c005c60 0x7f3464071cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.3:47002/0 (socket says 172.21.15.3:47002) 2024-05-21T17:41:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f3463fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34641ceb20 0x7f3464070eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49994/0 (socket says 172.21.15.3:49994) 2024-05-21T17:41:17.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.052+0000 7f3468b64640 1 -- 172.21.15.3:0/3567073679 learned_addr learned my addr 172.21.15.3:0/3567073679 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.053+0000 7f3469365640 1 --2- 172.21.15.3:0/3567073679 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464072ec0 0x7f34641da920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.053+0000 7f3468b64640 1 -- 172.21.15.3:0/3567073679 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464072ec0 msgr2=0x7f34641da920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.053+0000 7f3468b64640 1 --2- 172.21.15.3:0/3567073679 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464072ec0 0x7f34641da920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.053+0000 7f3468b64640 1 -- 172.21.15.3:0/3567073679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34641ceb20 msgr2=0x7f3464070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.053+0000 7f3468b64640 1 --2- 172.21.15.3:0/3567073679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34641ceb20 0x7f3464070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.053+0000 7f3468b64640 1 -- 172.21.15.3:0/3567073679 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3464072750 con 0x7f344c005c60 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.053+0000 7f3463fff640 1 --2- 172.21.15.3:0/3567073679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34641ceb20 0x7f3464070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f3468b64640 1 --2- 172.21.15.3:0/3567073679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c005c60 0x7f3464071cc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f345403c040 tx=0x7f3454004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f34617fa640 1 -- 172.21.15.3:0/3567073679 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454049030 con 0x7f344c005c60 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f34617fa640 1 -- 172.21.15.3:0/3567073679 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3454030540 con 0x7f344c005c60 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f346adc8640 1 -- 172.21.15.3:0/3567073679 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34641d73b0 con 0x7f344c005c60 2024-05-21T17:41:17.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f34617fa640 1 -- 172.21.15.3:0/3567073679 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345400f9f0 con 0x7f344c005c60 2024-05-21T17:41:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f346adc8640 1 -- 172.21.15.3:0/3567073679 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34641cdc30 con 0x7f344c005c60 2024-05-21T17:41:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f34617fa640 1 -- 172.21.15.3:0/3567073679 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f345404c070 con 0x7f344c005c60 2024-05-21T17:41:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f34617fa640 20 mgrc handle_mgr_map mgrmap(e 1) 2024-05-21T17:41:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f34617fa640 4 mgrc handle_mgr_map Got map version 1 2024-05-21T17:41:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f34617fa640 4 mgrc handle_mgr_map Active mgr is now 2024-05-21T17:41:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f34617fa640 4 mgrc reconnect No active mgr available yet 2024-05-21T17:41:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.054+0000 7f34617fa640 1 -- 172.21.15.3:0/3567073679 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f34540456f0 con 0x7f344c005c60 2024-05-21T17:41:17.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.055+0000 7f346adc8640 1 -- 172.21.15.3:0/3567073679 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3464069a90 con 0x7f344c005c60 2024-05-21T17:41:17.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.057+0000 7f34617fa640 1 -- 172.21.15.3:0/3567073679 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3454053070 con 0x7f344c005c60 2024-05-21T17:41:17.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.147+0000 7f346adc8640 1 -- 172.21.15.3:0/3567073679 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3464145a50 con 0x7f344c005c60 2024-05-21T17:41:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.155+0000 7f34617fa640 1 -- 172.21.15.3:0/3567073679 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f34540306f0 con 0x7f344c005c60 2024-05-21T17:41:17.157 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-05-21T17:41:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.156+0000 7f346adc8640 10 mgrc shutdown 2024-05-21T17:41:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.156+0000 7f346adc8640 1 -- 172.21.15.3:0/3567073679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c005c60 msgr2=0x7f3464071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.156+0000 7f346adc8640 1 --2- 172.21.15.3:0/3567073679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c005c60 0x7f3464071cc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f345403c040 tx=0x7f3454004730 comp rx=0 tx=0).stop 2024-05-21T17:41:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.156+0000 7f346adc8640 1 -- 172.21.15.3:0/3567073679 shutdown_connections 2024-05-21T17:41:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.156+0000 7f346adc8640 1 --2- 172.21.15.3:0/3567073679 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3464072ec0 0x7f34641da920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.156+0000 7f346adc8640 1 --2- 172.21.15.3:0/3567073679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f34641ceb20 0x7f3464070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.156+0000 7f346adc8640 1 --2- 172.21.15.3:0/3567073679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c005c60 0x7f3464071cc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.156+0000 7f346adc8640 1 -- 172.21.15.3:0/3567073679 >> 172.21.15.3:0/3567073679 conn(0x7f346406ccf0 msgr2=0x7f346406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.156+0000 7f346adc8640 1 -- 172.21.15.3:0/3567073679 shutdown_connections 2024-05-21T17:41:17.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:17.156+0000 7f346adc8640 1 -- 172.21.15.3:0/3567073679 wait complete. 2024-05-21T17:41:17.159 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:17.157+0000 7f3819eda040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T17:41:17.159 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:17.157+0000 7ff3a7c0f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T17:41:17.169 INFO:tasks.ceph:check_enable_crimson: False 2024-05-21T17:41:17.169 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-21T17:41:17.169 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:41:17.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-21T17:41:17.184 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:17.182+0000 7f8bf81b8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T17:41:17.196 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:41:17.196 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-21T17:41:17.221 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:41:17.221 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-21T17:41:17.286 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-05-21T17:41:17.286 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-21T17:41:17.322 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:17.321+0000 7f3819eda040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:41:17.322 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:17.321+0000 7ff3a7c0f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:41:17.346 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new a7ac7071-219d-4cf4-9f8a-977f495fd0d9 0 2024-05-21T17:41:17.348 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:17.346+0000 7f8bf81b8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:41:17.379 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:17.378+0000 7ff3a7c0f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T17:41:17.379 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:17.378+0000 7f3819eda040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T17:41:17.403 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:17.402+0000 7f8bf81b8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T17:41:17.480 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.479+0000 7f30ad7ce640 1 Processor -- start 2024-05-21T17:41:17.481 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.479+0000 7f30ad7ce640 1 -- start start 2024-05-21T17:41:17.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.480+0000 7f30ad7ce640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f30a8143a90 0x7f30a80a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.480+0000 7f30ad7ce640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30a80a7600 con 0x7f30a814c9b0 2024-05-21T17:41:17.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.480+0000 7f30ad7ce640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30a80b9cd0 con 0x7f30a80a9ed0 2024-05-21T17:41:17.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.480+0000 7f30ad7ce640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f30a80a6960 con 0x7f30a8143a90 2024-05-21T17:41:17.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.480+0000 7f30a6ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f30a8143a90 0x7f30a80a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.480+0000 7f30a67fc640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f30a80a9ed0 0x7f30a80a6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:44986/0 (socket says 172.21.15.203:44986) 2024-05-21T17:41:17.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.480+0000 7f30a67fc640 1 -- 172.21.15.203:0/3725587739 learned_addr learned my addr 172.21.15.203:0/3725587739 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-05-21T17:41:17.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.481+0000 7f30a6ffd640 1 -- 172.21.15.203:0/3725587739 >> v1:172.21.15.203:6789/0 conn(0x7f30a80a9ed0 legacy=0x7f30a80a6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.481+0000 7f30a6ffd640 1 -- 172.21.15.203:0/3725587739 >> v1:172.21.15.3:6789/0 conn(0x7f30a814c9b0 legacy=0x7f30a814eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.482 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.481+0000 7f30a6ffd640 1 -- 172.21.15.203:0/3725587739 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a80a7db0 con 0x7f30a8143a90 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.482+0000 7f30a6ffd640 1 --2- 172.21.15.203:0/3725587739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 0x7f30a80a82b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f30980097a0 tx=0x7f309802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1041c2dfd9bb4714 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.482+0000 7f30a5ffb640 1 -- 172.21.15.203:0/3725587739 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098035020 con 0x7f30a8143a90 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.482+0000 7f30a5ffb640 1 -- 172.21.15.203:0/3725587739 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3098031830 con 0x7f30a8143a90 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.482+0000 7f30a5ffb640 1 -- 172.21.15.203:0/3725587739 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098031c00 con 0x7f30a8143a90 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.482+0000 7f30ad7ce640 1 -- 172.21.15.203:0/3725587739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 msgr2=0x7f30a80a82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.482+0000 7f30ad7ce640 1 --2- 172.21.15.203:0/3725587739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 0x7f30a80a82b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f30980097a0 tx=0x7f309802b8d0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.482+0000 7f30ad7ce640 1 -- 172.21.15.203:0/3725587739 shutdown_connections 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.482+0000 7f30ad7ce640 1 --2- 172.21.15.203:0/3725587739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 0x7f30a80a82b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.482+0000 7f30ad7ce640 1 -- 172.21.15.203:0/3725587739 >> 172.21.15.203:0/3725587739 conn(0x7f30a8072640 msgr2=0x7f30a806c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.483+0000 7f30ad7ce640 1 -- 172.21.15.203:0/3725587739 shutdown_connections 2024-05-21T17:41:17.484 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.483+0000 7f30ad7ce640 1 -- 172.21.15.203:0/3725587739 wait complete. 2024-05-21T17:41:17.485 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.484+0000 7f30ad7ce640 1 Processor -- start 2024-05-21T17:41:17.485 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.484+0000 7f30ad7ce640 1 -- start start 2024-05-21T17:41:17.488 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30ad7ce640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30a80a9ed0 0x7f30a80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.488 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30ad7ce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 0x7f30a80a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.488 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30ad7ce640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30a814c9b0 0x7f30a81d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.488 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30ad7ce640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f30a8143e70 con 0x7f30a814c9b0 2024-05-21T17:41:17.488 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30ad7ce640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f30a80a87f0 con 0x7f30a8143a90 2024-05-21T17:41:17.488 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30ad7ce640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f30a8143ff0 con 0x7f30a80a9ed0 2024-05-21T17:41:17.488 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30a67fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 0x7f30a80a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30a67fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 0x7f30a80a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59348/0 (socket says 172.21.15.203:59348) 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30a67fc640 1 -- 172.21.15.203:0/1200186799 learned_addr learned my addr 172.21.15.203:0/1200186799 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30a6ffd640 1 --2- 172.21.15.203:0/1200186799 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30a80a9ed0 0x7f30a80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.485+0000 7f30a77fe640 1 --2- 172.21.15.203:0/1200186799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30a814c9b0 0x7f30a81d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f30a67fc640 1 -- 172.21.15.203:0/1200186799 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30a80a9ed0 msgr2=0x7f30a80a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f30a67fc640 1 --2- 172.21.15.203:0/1200186799 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30a80a9ed0 0x7f30a80a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f30a67fc640 1 -- 172.21.15.203:0/1200186799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30a814c9b0 msgr2=0x7f30a81d9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f30a67fc640 1 --2- 172.21.15.203:0/1200186799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30a814c9b0 0x7f30a81d9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f30a67fc640 1 -- 172.21.15.203:0/1200186799 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a81470e0 con 0x7f30a8143a90 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f30a67fc640 1 --2- 172.21.15.203:0/1200186799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 0x7f30a80a3c30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c500 tx=0x7f309c00c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f308f7fe640 1 -- 172.21.15.203:0/1200186799 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c017070 con 0x7f30a8143a90 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f308f7fe640 1 -- 172.21.15.203:0/1200186799 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f309c005ce0 con 0x7f30a8143a90 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f30ad7ce640 1 -- 172.21.15.203:0/1200186799 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a81daa30 con 0x7f30a8143a90 2024-05-21T17:41:17.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f308f7fe640 1 -- 172.21.15.203:0/1200186799 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c0072f0 con 0x7f30a8143a90 2024-05-21T17:41:17.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f30ad7ce640 1 -- 172.21.15.203:0/1200186799 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a81397b0 con 0x7f30a8143a90 2024-05-21T17:41:17.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f308f7fe640 1 -- 172.21.15.203:0/1200186799 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f309c01a070 con 0x7f30a8143a90 2024-05-21T17:41:17.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f308f7fe640 20 mgrc handle_mgr_map mgrmap(e 1) 2024-05-21T17:41:17.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f308f7fe640 4 mgrc handle_mgr_map Got map version 1 2024-05-21T17:41:17.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f308f7fe640 4 mgrc handle_mgr_map Active mgr is now 2024-05-21T17:41:17.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f308f7fe640 4 mgrc reconnect No active mgr available yet 2024-05-21T17:41:17.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.486+0000 7f308f7fe640 1 -- 172.21.15.203:0/1200186799 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f309c012740 con 0x7f30a8143a90 2024-05-21T17:41:17.491 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.487+0000 7f30ad7ce640 1 -- 172.21.15.203:0/1200186799 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a80a9ce0 con 0x7f30a8143a90 2024-05-21T17:41:17.491 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.490+0000 7f308f7fe640 1 -- 172.21.15.203:0/1200186799 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f309c007490 con 0x7f30a8143a90 2024-05-21T17:41:17.564 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.563+0000 7f30ad7ce640 1 -- 172.21.15.203:0/1200186799 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a7ac7071-219d-4cf4-9f8a-977f495fd0d9", "id": 0} v 0) -- 0x7f30a80a6440 con 0x7f30a8143a90 2024-05-21T17:41:17.572 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.571+0000 7f308f7fe640 1 -- 172.21.15.203:0/1200186799 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7ac7071-219d-4cf4-9f8a-977f495fd0d9", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f309c01cbc0 con 0x7f30a8143a90 2024-05-21T17:41:17.572 INFO:teuthology.orchestra.run.smithi203.stdout:0 2024-05-21T17:41:17.573 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.572+0000 7f30ad7ce640 10 mgrc shutdown 2024-05-21T17:41:17.573 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.572+0000 7f30ad7ce640 1 -- 172.21.15.203:0/1200186799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 msgr2=0x7f30a80a3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.574 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.572+0000 7f30ad7ce640 1 --2- 172.21.15.203:0/1200186799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 0x7f30a80a3c30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f309c00c500 tx=0x7f309c00c9c0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.574 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.572+0000 7f30ad7ce640 1 -- 172.21.15.203:0/1200186799 shutdown_connections 2024-05-21T17:41:17.574 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.572+0000 7f30ad7ce640 1 --2- 172.21.15.203:0/1200186799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f30a814c9b0 0x7f30a81d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.574 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.572+0000 7f30ad7ce640 1 --2- 172.21.15.203:0/1200186799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30a8143a90 0x7f30a80a3c30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.574 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.572+0000 7f30ad7ce640 1 --2- 172.21.15.203:0/1200186799 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f30a80a9ed0 0x7f30a80a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.574 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.572+0000 7f30ad7ce640 1 -- 172.21.15.203:0/1200186799 >> 172.21.15.203:0/1200186799 conn(0x7f30a8072640 msgr2=0x7f30a80b7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:17.574 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.572+0000 7f30ad7ce640 1 -- 172.21.15.203:0/1200186799 shutdown_connections 2024-05-21T17:41:17.574 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.572+0000 7f30ad7ce640 1 -- 172.21.15.203:0/1200186799 wait complete. 2024-05-21T17:41:17.584 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 89df5105-5900-4ffa-b8c1-6f480865a085 1 2024-05-21T17:41:17.652 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:17.650+0000 7ff3a7c0f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T17:41:17.652 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:17.650+0000 7f3819eda040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T17:41:17.681 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:17.678+0000 7f8bf81b8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T17:41:17.684 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.682+0000 7f2e4677e640 1 Processor -- start 2024-05-21T17:41:17.684 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.683+0000 7f2e4677e640 1 -- start start 2024-05-21T17:41:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e4677e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e4014f9e0 0x7f2e40151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e4677e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e400ad810 con 0x7f2e400ad430 2024-05-21T17:41:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e4677e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e40146990 con 0x7f2e40145ba0 2024-05-21T17:41:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e4677e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f2e40146c30 con 0x7f2e4014f9e0 2024-05-21T17:41:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e3ffff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f2e40145ba0 0x7f2e4006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:44992/0 (socket says 172.21.15.203:44992) 2024-05-21T17:41:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e3ffff640 1 -- 172.21.15.203:0/1945330992 learned_addr learned my addr 172.21.15.203:0/1945330992 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-05-21T17:41:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e44d1b640 1 --2- 172.21.15.203:0/1945330992 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e4014f9e0 0x7f2e40151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e3effd640 1 -- 172.21.15.203:0/1945330992 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2406217456 0 0) 0x7f2e40146990 con 0x7f2e40145ba0 2024-05-21T17:41:17.686 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e3effd640 1 -- 172.21.15.203:0/1945330992 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e28003520 con 0x7f2e40145ba0 2024-05-21T17:41:17.687 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e3effd640 1 -- 172.21.15.203:0/1945330992 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044792278 0 0) 0x7f2e400ad810 con 0x7f2e400ad430 2024-05-21T17:41:17.687 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e3effd640 1 -- 172.21.15.203:0/1945330992 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e40146990 con 0x7f2e400ad430 2024-05-21T17:41:17.687 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e44d1b640 1 -- 172.21.15.203:0/1945330992 >> v1:172.21.15.203:6789/0 conn(0x7f2e40145ba0 legacy=0x7f2e4006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.687 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e44d1b640 1 -- 172.21.15.203:0/1945330992 >> v1:172.21.15.3:6789/0 conn(0x7f2e400ad430 legacy=0x7f2e40145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.687 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.684+0000 7f2e44d1b640 1 -- 172.21.15.203:0/1945330992 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e40146690 con 0x7f2e4014f9e0 2024-05-21T17:41:17.688 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.685+0000 7f2e44d1b640 1 --2- 172.21.15.203:0/1945330992 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e4014f9e0 0x7f2e40151dd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009dc0 tx=0x7f2e3802bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95d3024a492d2e43 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:17.688 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.686+0000 7f2e3effd640 1 -- 172.21.15.203:0/1945330992 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38039070 con 0x7f2e4014f9e0 2024-05-21T17:41:17.688 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.686+0000 7f2e3effd640 1 -- 172.21.15.203:0/1945330992 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3802cd50 con 0x7f2e4014f9e0 2024-05-21T17:41:17.688 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.686+0000 7f2e3effd640 1 -- 172.21.15.203:0/1945330992 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38034660 con 0x7f2e4014f9e0 2024-05-21T17:41:17.688 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.687+0000 7f2e4677e640 1 -- 172.21.15.203:0/1945330992 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e4014f9e0 msgr2=0x7f2e40151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.687+0000 7f2e4677e640 1 --2- 172.21.15.203:0/1945330992 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e4014f9e0 0x7f2e40151dd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009dc0 tx=0x7f2e3802bfa0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.687+0000 7f2e4677e640 1 -- 172.21.15.203:0/1945330992 shutdown_connections 2024-05-21T17:41:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.687+0000 7f2e4677e640 1 --2- 172.21.15.203:0/1945330992 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e4014f9e0 0x7f2e40151dd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.687+0000 7f2e4677e640 1 -- 172.21.15.203:0/1945330992 >> 172.21.15.203:0/1945330992 conn(0x7f2e400bbc20 msgr2=0x7f2e400bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.687+0000 7f2e4677e640 1 -- 172.21.15.203:0/1945330992 shutdown_connections 2024-05-21T17:41:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.687+0000 7f2e4677e640 1 -- 172.21.15.203:0/1945330992 wait complete. 2024-05-21T17:41:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.688+0000 7f2e4677e640 1 Processor -- start 2024-05-21T17:41:17.689 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.689+0000 7f2e4677e640 1 -- start start 2024-05-21T17:41:17.693 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.689+0000 7f2e4677e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e400ad430 0x7f2e40149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.693 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.689+0000 7f2e4677e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e40145ba0 0x7f2e4013fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.693 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.689+0000 7f2e4677e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e4014f9e0 0x7f2e400a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.693 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.689+0000 7f2e4677e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e400bc710 con 0x7f2e40145ba0 2024-05-21T17:41:17.693 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.689+0000 7f2e4677e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e400adcc0 con 0x7f2e4014f9e0 2024-05-21T17:41:17.693 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.689+0000 7f2e4677e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f2e400bc890 con 0x7f2e400ad430 2024-05-21T17:41:17.693 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e44d1b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e4014f9e0 0x7f2e400a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.693 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e3ffff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e400ad430 0x7f2e40149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.693 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e44d1b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e4014f9e0 0x7f2e400a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59364/0 (socket says 172.21.15.203:59364) 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e44d1b640 1 -- 172.21.15.203:0/3022946373 learned_addr learned my addr 172.21.15.203:0/3022946373 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e3f7fe640 1 --2- 172.21.15.203:0/3022946373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e40145ba0 0x7f2e4013fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e44d1b640 1 -- 172.21.15.203:0/3022946373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e400ad430 msgr2=0x7f2e40149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e44d1b640 1 --2- 172.21.15.203:0/3022946373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e400ad430 0x7f2e40149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e44d1b640 1 -- 172.21.15.203:0/3022946373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e40145ba0 msgr2=0x7f2e4013fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e44d1b640 1 --2- 172.21.15.203:0/3022946373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e40145ba0 0x7f2e4013fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e44d1b640 1 -- 172.21.15.203:0/3022946373 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e40140000 con 0x7f2e4014f9e0 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e3ffff640 1 --2- 172.21.15.203:0/3022946373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e400ad430 0x7f2e40149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e44d1b640 1 --2- 172.21.15.203:0/3022946373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e4014f9e0 0x7f2e400a5980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009eb0 tx=0x7f2e38004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e3cff9640 1 -- 172.21.15.203:0/3022946373 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38039050 con 0x7f2e4014f9e0 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e3cff9640 1 -- 172.21.15.203:0/3022946373 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3803a040 con 0x7f2e4014f9e0 2024-05-21T17:41:17.694 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e4677e640 1 -- 172.21.15.203:0/3022946373 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e400a6610 con 0x7f2e4014f9e0 2024-05-21T17:41:17.695 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.690+0000 7f2e3cff9640 1 -- 172.21.15.203:0/3022946373 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38009450 con 0x7f2e4014f9e0 2024-05-21T17:41:17.695 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.691+0000 7f2e4677e640 1 -- 172.21.15.203:0/3022946373 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4014a3f0 con 0x7f2e4014f9e0 2024-05-21T17:41:17.695 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.691+0000 7f2e3cff9640 1 -- 172.21.15.203:0/3022946373 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e38047020 con 0x7f2e4014f9e0 2024-05-21T17:41:17.695 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.691+0000 7f2e3cff9640 20 mgrc handle_mgr_map mgrmap(e 1) 2024-05-21T17:41:17.695 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.691+0000 7f2e3cff9640 4 mgrc handle_mgr_map Got map version 1 2024-05-21T17:41:17.695 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.691+0000 7f2e3cff9640 4 mgrc handle_mgr_map Active mgr is now 2024-05-21T17:41:17.695 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.691+0000 7f2e3cff9640 4 mgrc reconnect No active mgr available yet 2024-05-21T17:41:17.695 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.691+0000 7f2e3cff9640 1 -- 172.21.15.203:0/3022946373 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f2e3803e610 con 0x7f2e4014f9e0 2024-05-21T17:41:17.695 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.692+0000 7f2e4677e640 1 -- 172.21.15.203:0/3022946373 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e40069a90 con 0x7f2e4014f9e0 2024-05-21T17:41:17.695 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.694+0000 7f2e3cff9640 1 -- 172.21.15.203:0/3022946373 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2e3804e020 con 0x7f2e4014f9e0 2024-05-21T17:41:17.701 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:17.701+0000 7f3819eda040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T17:41:17.703 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:17.702+0000 7ff3a7c0f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T17:41:17.728 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:17.726+0000 7f8bf81b8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T17:41:17.768 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.767+0000 7f2e4677e640 1 -- 172.21.15.203:0/3022946373 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "89df5105-5900-4ffa-b8c1-6f480865a085", "id": 1} v 0) -- 0x7f2e40143d10 con 0x7f2e4014f9e0 2024-05-21T17:41:17.776 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.775+0000 7f2e3cff9640 1 -- 172.21.15.203:0/3022946373 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "89df5105-5900-4ffa-b8c1-6f480865a085", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f2e38004800 con 0x7f2e4014f9e0 2024-05-21T17:41:17.776 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-05-21T17:41:17.777 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.776+0000 7f2e4677e640 10 mgrc shutdown 2024-05-21T17:41:17.777 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.776+0000 7f2e4677e640 1 -- 172.21.15.203:0/3022946373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e4014f9e0 msgr2=0x7f2e400a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.777 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.776+0000 7f2e4677e640 1 --2- 172.21.15.203:0/3022946373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e4014f9e0 0x7f2e400a5980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009eb0 tx=0x7f2e38004a40 comp rx=0 tx=0).stop 2024-05-21T17:41:17.777 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.776+0000 7f2e4677e640 1 -- 172.21.15.203:0/3022946373 shutdown_connections 2024-05-21T17:41:17.777 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.776+0000 7f2e4677e640 1 --2- 172.21.15.203:0/3022946373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e4014f9e0 0x7f2e400a5980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.777 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.776+0000 7f2e4677e640 1 --2- 172.21.15.203:0/3022946373 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e40145ba0 0x7f2e4013fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.778 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.776+0000 7f2e4677e640 1 --2- 172.21.15.203:0/3022946373 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2e400ad430 0x7f2e40149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.778 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.776+0000 7f2e4677e640 1 -- 172.21.15.203:0/3022946373 >> 172.21.15.203:0/3022946373 conn(0x7f2e400bbc20 msgr2=0x7f2e400a9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:17.778 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.776+0000 7f2e4677e640 1 -- 172.21.15.203:0/3022946373 shutdown_connections 2024-05-21T17:41:17.778 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.776+0000 7f2e4677e640 1 -- 172.21.15.203:0/3022946373 wait complete. 2024-05-21T17:41:17.789 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 15f83334-8efc-40c2-af0c-eeed45923086 2 2024-05-21T17:41:17.890 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.887+0000 7f9971b93640 1 Processor -- start 2024-05-21T17:41:17.890 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.888+0000 7f9971b93640 1 -- start start 2024-05-21T17:41:17.890 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.888+0000 7f9971b93640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f996c0aa550 0x7f996c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.888+0000 7f9971b93640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996c0a91b0 con 0x7f996c06d520 2024-05-21T17:41:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.888+0000 7f9971b93640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996c0a6ad0 con 0x7f996c0a9e60 2024-05-21T17:41:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.888+0000 7f9971b93640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f996c0a6d90 con 0x7f996c0aa550 2024-05-21T17:41:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.888+0000 7f996affd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f996c0a9e60 0x7f996c14af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:45004/0 (socket says 172.21.15.203:45004) 2024-05-21T17:41:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.888+0000 7f996affd640 1 -- 172.21.15.203:0/3356275129 learned_addr learned my addr 172.21.15.203:0/3356275129 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-05-21T17:41:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.888+0000 7f996bfff640 1 --2- 172.21.15.203:0/3356275129 >> v2:172.21.15.3:3300/0 conn(0x7f996c0aa550 0x7f996c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.889+0000 7f996a7fc640 1 -- 172.21.15.203:0/3356275129 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 927112219 0 0) 0x7f996c0a6ad0 con 0x7f996c0a9e60 2024-05-21T17:41:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.889+0000 7f996a7fc640 1 -- 172.21.15.203:0/3356275129 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9954003440 con 0x7f996c0a9e60 2024-05-21T17:41:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.889+0000 7f996bfff640 1 -- 172.21.15.203:0/3356275129 >> v1:172.21.15.203:6789/0 conn(0x7f996c0a9e60 legacy=0x7f996c14af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.891 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.889+0000 7f996bfff640 1 -- 172.21.15.203:0/3356275129 >> v1:172.21.15.3:6789/0 conn(0x7f996c06d520 legacy=0x7f996c0a8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.892 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.889+0000 7f996a7fc640 1 -- 172.21.15.203:0/3356275129 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333226874 0 0) 0x7f996c0a91b0 con 0x7f996c06d520 2024-05-21T17:41:17.892 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.889+0000 7f996bfff640 1 -- 172.21.15.203:0/3356275129 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c0a5f20 con 0x7f996c0aa550 2024-05-21T17:41:17.893 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.890+0000 7f996bfff640 1 --2- 172.21.15.203:0/3356275129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c0aa550 0x7f996c0a63a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9964009d90 tx=0x7f996402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f7c08c8f27ddf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:17.893 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.890+0000 7f996a7fc640 1 -- 172.21.15.203:0/3356275129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964039070 con 0x7f996c0aa550 2024-05-21T17:41:17.894 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.890+0000 7f996a7fc640 1 -- 172.21.15.203:0/3356275129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996402cc80 con 0x7f996c0aa550 2024-05-21T17:41:17.894 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.890+0000 7f996a7fc640 1 -- 172.21.15.203:0/3356275129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964034660 con 0x7f996c0aa550 2024-05-21T17:41:17.894 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.891+0000 7f9971b93640 1 -- 172.21.15.203:0/3356275129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c0aa550 msgr2=0x7f996c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.894 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.891+0000 7f9971b93640 1 --2- 172.21.15.203:0/3356275129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c0aa550 0x7f996c0a63a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9964009d90 tx=0x7f996402bed0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.894 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.891+0000 7f9971b93640 1 -- 172.21.15.203:0/3356275129 shutdown_connections 2024-05-21T17:41:17.894 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.891+0000 7f9971b93640 1 --2- 172.21.15.203:0/3356275129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c0aa550 0x7f996c0a63a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.894 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.891+0000 7f9971b93640 1 -- 172.21.15.203:0/3356275129 >> 172.21.15.203:0/3356275129 conn(0x7f996c070df0 msgr2=0x7f996c0a74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:17.894 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.891+0000 7f9971b93640 1 -- 172.21.15.203:0/3356275129 shutdown_connections 2024-05-21T17:41:17.894 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.891+0000 7f9971b93640 1 -- 172.21.15.203:0/3356275129 wait complete. 2024-05-21T17:41:17.895 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.892+0000 7f9971b93640 1 Processor -- start 2024-05-21T17:41:17.895 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.892+0000 7f9971b93640 1 -- start start 2024-05-21T17:41:17.897 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f9971b93640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f996c06d520 0x7f996c1ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.897 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f9971b93640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f996c0a9e60 0x7f996c1dabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.897 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f9971b93640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c0aa550 0x7f996c1db0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:17.897 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f9971b93640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f996c138d00 con 0x7f996c0aa550 2024-05-21T17:41:17.897 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f9971b93640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f996c06d900 con 0x7f996c06d520 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f9971b93640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f996c138e80 con 0x7f996c0a9e60 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996b7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f996c06d520 0x7f996c1ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996b7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f996c06d520 0x7f996c1ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59366/0 (socket says 172.21.15.203:59366) 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996b7fe640 1 -- 172.21.15.203:0/154108197 learned_addr learned my addr 172.21.15.203:0/154108197 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996bfff640 1 --2- 172.21.15.203:0/154108197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c0aa550 0x7f996c1db0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996affd640 1 --2- 172.21.15.203:0/154108197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f996c0a9e60 0x7f996c1dabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996b7fe640 1 -- 172.21.15.203:0/154108197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f996c0a9e60 msgr2=0x7f996c1dabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996b7fe640 1 --2- 172.21.15.203:0/154108197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f996c0a9e60 0x7f996c1dabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996b7fe640 1 -- 172.21.15.203:0/154108197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c0aa550 msgr2=0x7f996c1db0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996b7fe640 1 --2- 172.21.15.203:0/154108197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c0aa550 0x7f996c1db0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996b7fe640 1 -- 172.21.15.203:0/154108197 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c12f6d0 con 0x7f996c06d520 2024-05-21T17:41:17.898 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f996b7fe640 1 --2- 172.21.15.203:0/154108197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f996c06d520 0x7f996c1ccc90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c500 tx=0x7f995c00c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.893+0000 7f9953fff640 1 -- 172.21.15.203:0/154108197 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c017070 con 0x7f996c06d520 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.894+0000 7f9953fff640 1 -- 172.21.15.203:0/154108197 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c0035f0 con 0x7f996c06d520 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.894+0000 7f9953fff640 1 -- 172.21.15.203:0/154108197 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c0039e0 con 0x7f996c06d520 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.894+0000 7f9971b93640 1 -- 172.21.15.203:0/154108197 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c132080 con 0x7f996c06d520 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.894+0000 7f9971b93640 1 -- 172.21.15.203:0/154108197 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c1cd1d0 con 0x7f996c06d520 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.894+0000 7f9953fff640 1 -- 172.21.15.203:0/154108197 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f995c01a070 con 0x7f996c06d520 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.894+0000 7f9953fff640 20 mgrc handle_mgr_map mgrmap(e 1) 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.894+0000 7f9953fff640 4 mgrc handle_mgr_map Got map version 1 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.894+0000 7f9953fff640 4 mgrc handle_mgr_map Active mgr is now 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.894+0000 7f9953fff640 4 mgrc reconnect No active mgr available yet 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.894+0000 7f9953fff640 1 -- 172.21.15.203:0/154108197 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f995c012820 con 0x7f996c06d520 2024-05-21T17:41:17.899 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.895+0000 7f9971b93640 1 -- 172.21.15.203:0/154108197 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f996c069290 con 0x7f996c06d520 2024-05-21T17:41:17.900 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.897+0000 7f9953fff640 1 -- 172.21.15.203:0/154108197 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f995c021070 con 0x7f996c06d520 2024-05-21T17:41:17.971 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.970+0000 7f9971b93640 1 -- 172.21.15.203:0/154108197 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "15f83334-8efc-40c2-af0c-eeed45923086", "id": 2} v 0) -- 0x7f996c0a93c0 con 0x7f996c06d520 2024-05-21T17:41:17.979 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.978+0000 7f9953fff640 1 -- 172.21.15.203:0/154108197 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "15f83334-8efc-40c2-af0c-eeed45923086", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f995c003dc0 con 0x7f996c06d520 2024-05-21T17:41:17.979 INFO:teuthology.orchestra.run.smithi203.stdout:2 2024-05-21T17:41:17.980 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.979+0000 7f9971b93640 10 mgrc shutdown 2024-05-21T17:41:17.980 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.979+0000 7f9971b93640 1 -- 172.21.15.203:0/154108197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f996c06d520 msgr2=0x7f996c1ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:17.980 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.979+0000 7f9971b93640 1 --2- 172.21.15.203:0/154108197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f996c06d520 0x7f996c1ccc90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c500 tx=0x7f995c00c9c0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.981 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.979+0000 7f9971b93640 1 -- 172.21.15.203:0/154108197 shutdown_connections 2024-05-21T17:41:17.981 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.979+0000 7f9971b93640 1 --2- 172.21.15.203:0/154108197 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f996c0aa550 0x7f996c1db0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.981 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.979+0000 7f9971b93640 1 --2- 172.21.15.203:0/154108197 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f996c0a9e60 0x7f996c1dabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.981 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.979+0000 7f9971b93640 1 --2- 172.21.15.203:0/154108197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f996c06d520 0x7f996c1ccc90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:17.981 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.979+0000 7f9971b93640 1 -- 172.21.15.203:0/154108197 >> 172.21.15.203:0/154108197 conn(0x7f996c070df0 msgr2=0x7f996c0a6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:17.981 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.979+0000 7f9971b93640 1 -- 172.21.15.203:0/154108197 shutdown_connections 2024-05-21T17:41:17.981 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:17.979+0000 7f9971b93640 1 -- 172.21.15.203:0/154108197 wait complete. 2024-05-21T17:41:17.991 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 6a27ac8e-f957-4455-b477-3090081766c4 3 2024-05-21T17:41:18.093 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.092+0000 7fe7d4a80640 1 Processor -- start 2024-05-21T17:41:18.094 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.092+0000 7fe7d4a80640 1 -- start start 2024-05-21T17:41:18.096 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.093+0000 7fe7d4a80640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fe7d00752d0 0x7fe7d00a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.096 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.093+0000 7fe7d4a80640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe7d013a700 0x7fe7d0071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.093+0000 7fe7d4a80640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7d0147660 con 0x7fe7d00756b0 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.093+0000 7fe7d4a80640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d0147920 con 0x7fe7d00752d0 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.093+0000 7fe7d4a80640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d0148480 con 0x7fe7d013a700 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.093+0000 7fe7ced9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe7d013a700 0x7fe7d0071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.093+0000 7fe7cdd9b640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe7d00756b0 0x7fe7d0147d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:45018/0 (socket says 172.21.15.203:45018) 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.093+0000 7fe7cdd9b640 1 -- 172.21.15.203:0/148744623 learned_addr learned my addr 172.21.15.203:0/148744623 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.093+0000 7fe7ce59c640 1 --2- 172.21.15.203:0/148744623 >> v2:172.21.15.3:3300/0 conn(0x7fe7d00752d0 0x7fe7d00a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.094+0000 7fe7ced9d640 1 -- 172.21.15.203:0/148744623 >> v2:172.21.15.3:3300/0 conn(0x7fe7d00752d0 msgr2=0x7fe7d00a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.094+0000 7fe7ced9d640 1 --2- 172.21.15.203:0/148744623 >> v2:172.21.15.3:3300/0 conn(0x7fe7d00752d0 0x7fe7d00a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.094+0000 7fe7ced9d640 1 -- 172.21.15.203:0/148744623 >> v1:172.21.15.203:6789/0 conn(0x7fe7d00756b0 legacy=0x7fe7d0147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.097 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.094+0000 7fe7ced9d640 1 -- 172.21.15.203:0/148744623 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d0071aa0 con 0x7fe7d013a700 2024-05-21T17:41:18.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.094+0000 7fe7ced9d640 1 --2- 172.21.15.203:0/148744623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d013a700 0x7fe7d0071fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00a73d0 tx=0x7fe7c802be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5f22e06e59158c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:18.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.095+0000 7fe7cd59a640 1 -- 172.21.15.203:0/148744623 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8039070 con 0x7fe7d013a700 2024-05-21T17:41:18.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.095+0000 7fe7cd59a640 1 -- 172.21.15.203:0/148744623 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c802cbf0 con 0x7fe7d013a700 2024-05-21T17:41:18.098 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.095+0000 7fe7cd59a640 1 -- 172.21.15.203:0/148744623 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8034440 con 0x7fe7d013a700 2024-05-21T17:41:18.099 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.096+0000 7fe7d4a80640 1 -- 172.21.15.203:0/148744623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d013a700 msgr2=0x7fe7d0071fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.099 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.096+0000 7fe7d4a80640 1 --2- 172.21.15.203:0/148744623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d013a700 0x7fe7d0071fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00a73d0 tx=0x7fe7c802be40 comp rx=0 tx=0).stop 2024-05-21T17:41:18.099 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.096+0000 7fe7d4a80640 1 -- 172.21.15.203:0/148744623 shutdown_connections 2024-05-21T17:41:18.099 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.096+0000 7fe7d4a80640 1 --2- 172.21.15.203:0/148744623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d013a700 0x7fe7d0071fa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.100 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.096+0000 7fe7d4a80640 1 --2- 172.21.15.203:0/148744623 >> v2:172.21.15.3:3300/0 conn(0x7fe7d00752d0 0x7fe7d00a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.100 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.096+0000 7fe7d4a80640 1 -- 172.21.15.203:0/148744623 >> 172.21.15.203:0/148744623 conn(0x7fe7d00a74a0 msgr2=0x7fe7d00a78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:18.100 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.096+0000 7fe7d4a80640 1 -- 172.21.15.203:0/148744623 shutdown_connections 2024-05-21T17:41:18.100 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.096+0000 7fe7d4a80640 1 -- 172.21.15.203:0/148744623 wait complete. 2024-05-21T17:41:18.100 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.098+0000 7fe7d4a80640 1 Processor -- start 2024-05-21T17:41:18.100 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.098+0000 7fe7d4a80640 1 -- start start 2024-05-21T17:41:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.098+0000 7fe7d4a80640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d00752d0 0x7fe7d0138870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7d4a80640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d013a700 0x7fe7d0131050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7d4a80640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d0131590 0x7fe7d0133e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7d4a80640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0072870 con 0x7fe7d013a700 2024-05-21T17:41:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7d4a80640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d00726f0 con 0x7fe7d0131590 2024-05-21T17:41:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7d4a80640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d00729f0 con 0x7fe7d00752d0 2024-05-21T17:41:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7ced9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d0131590 0x7fe7d0133e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7ced9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d0131590 0x7fe7d0133e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59398/0 (socket says 172.21.15.203:59398) 2024-05-21T17:41:18.103 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7ced9d640 1 -- 172.21.15.203:0/4217281134 learned_addr learned my addr 172.21.15.203:0/4217281134 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7cdd9b640 1 --2- 172.21.15.203:0/4217281134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d013a700 0x7fe7d0131050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7ce59c640 1 --2- 172.21.15.203:0/4217281134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d00752d0 0x7fe7d0138870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7ced9d640 1 -- 172.21.15.203:0/4217281134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d00752d0 msgr2=0x7fe7d0138870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7ced9d640 1 --2- 172.21.15.203:0/4217281134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d00752d0 0x7fe7d0138870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7ced9d640 1 -- 172.21.15.203:0/4217281134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d013a700 msgr2=0x7fe7d0131050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7ced9d640 1 --2- 172.21.15.203:0/4217281134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d013a700 0x7fe7d0131050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7ced9d640 1 -- 172.21.15.203:0/4217281134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d01349a0 con 0x7fe7d0131590 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7ced9d640 1 --2- 172.21.15.203:0/4217281134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d0131590 0x7fe7d0133e60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009e80 tx=0x7fe7c8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7beffd640 1 -- 172.21.15.203:0/4217281134 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8039050 con 0x7fe7d0131590 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7beffd640 1 -- 172.21.15.203:0/4217281134 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c8045070 con 0x7fe7d0131590 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7beffd640 1 -- 172.21.15.203:0/4217281134 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8009450 con 0x7fe7d0131590 2024-05-21T17:41:18.104 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.099+0000 7fe7d4a80640 1 -- 172.21.15.203:0/4217281134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d01dcb90 con 0x7fe7d0131590 2024-05-21T17:41:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.100+0000 7fe7d4a80640 1 -- 172.21.15.203:0/4217281134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d0133510 con 0x7fe7d0131590 2024-05-21T17:41:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.100+0000 7fe7beffd640 1 -- 172.21.15.203:0/4217281134 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe7c803e850 con 0x7fe7d0131590 2024-05-21T17:41:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.100+0000 7fe7beffd640 20 mgrc handle_mgr_map mgrmap(e 1) 2024-05-21T17:41:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.100+0000 7fe7beffd640 4 mgrc handle_mgr_map Got map version 1 2024-05-21T17:41:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.100+0000 7fe7beffd640 4 mgrc handle_mgr_map Active mgr is now 2024-05-21T17:41:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.100+0000 7fe7beffd640 4 mgrc reconnect No active mgr available yet 2024-05-21T17:41:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.100+0000 7fe7beffd640 1 -- 172.21.15.203:0/4217281134 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe7c8040450 con 0x7fe7d0131590 2024-05-21T17:41:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.100+0000 7fe7d4a80640 1 -- 172.21.15.203:0/4217281134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d0069290 con 0x7fe7d0131590 2024-05-21T17:41:18.105 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.102+0000 7fe7beffd640 1 -- 172.21.15.203:0/4217281134 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe7c803ecd0 con 0x7fe7d0131590 2024-05-21T17:41:18.178 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.177+0000 7fe7d4a80640 1 -- 172.21.15.203:0/4217281134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6a27ac8e-f957-4455-b477-3090081766c4", "id": 3} v 0) -- 0x7fe7d00b7900 con 0x7fe7d0131590 2024-05-21T17:41:18.182 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.181+0000 7fe7beffd640 1 -- 172.21.15.203:0/4217281134 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6a27ac8e-f957-4455-b477-3090081766c4", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe7c804a460 con 0x7fe7d0131590 2024-05-21T17:41:18.182 INFO:teuthology.orchestra.run.smithi203.stdout:3 2024-05-21T17:41:18.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.182+0000 7fe7d4a80640 10 mgrc shutdown 2024-05-21T17:41:18.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.182+0000 7fe7d4a80640 1 -- 172.21.15.203:0/4217281134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d0131590 msgr2=0x7fe7d0133e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.182+0000 7fe7d4a80640 1 --2- 172.21.15.203:0/4217281134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d0131590 0x7fe7d0133e60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009e80 tx=0x7fe7c8004a40 comp rx=0 tx=0).stop 2024-05-21T17:41:18.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.182+0000 7fe7d4a80640 1 -- 172.21.15.203:0/4217281134 shutdown_connections 2024-05-21T17:41:18.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.182+0000 7fe7d4a80640 1 --2- 172.21.15.203:0/4217281134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe7d0131590 0x7fe7d0133e60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.182+0000 7fe7d4a80640 1 --2- 172.21.15.203:0/4217281134 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d013a700 0x7fe7d0131050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.182+0000 7fe7d4a80640 1 --2- 172.21.15.203:0/4217281134 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe7d00752d0 0x7fe7d0138870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.182+0000 7fe7d4a80640 1 -- 172.21.15.203:0/4217281134 >> 172.21.15.203:0/4217281134 conn(0x7fe7d00a74a0 msgr2=0x7fe7d0147b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:18.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.182+0000 7fe7d4a80640 1 -- 172.21.15.203:0/4217281134 shutdown_connections 2024-05-21T17:41:18.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-05-21T17:41:18.182+0000 7fe7d4a80640 1 -- 172.21.15.203:0/4217281134 wait complete. 2024-05-21T17:41:18.194 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-21T17:41:18.194 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-21T17:41:18.196 INFO:tasks.ceph.osd.0:Started 2024-05-21T17:41:18.196 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-21T17:41:18.197 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-21T17:41:18.199 INFO:tasks.ceph.osd.1:Started 2024-05-21T17:41:18.199 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-21T17:41:18.199 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-21T17:41:18.201 INFO:tasks.ceph.osd.2:Started 2024-05-21T17:41:18.201 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-21T17:41:18.201 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-21T17:41:18.203 INFO:tasks.ceph.osd.3:Started 2024-05-21T17:41:18.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T17:41:18.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.306+0000 7f0d7f66c640 1 Processor -- start 2024-05-21T17:41:18.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.306+0000 7f0d7f66c640 1 -- start start 2024-05-21T17:41:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.306+0000 7f0d7f66c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d78144cb0 0x7f0d78073e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7f66c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0d780a6d10 0x7f0d781476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7f66c640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d780a7990 con 0x7f0d780a7180 2024-05-21T17:41:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7f66c640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0d781440d0 con 0x7f0d78144cb0 2024-05-21T17:41:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7f66c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f0d780a6b00 con 0x7f0d780a6d10 2024-05-21T17:41:18.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7d408640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0d78144cb0 0x7f0d78073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7dc09640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f0d780a7180 0x7f0d78147bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33190/0 (socket says 172.21.15.3:33190) 2024-05-21T17:41:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7dc09640 1 -- 172.21.15.3:0/1160782986 learned_addr learned my addr 172.21.15.3:0/1160782986 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7cc07640 1 --2- 172.21.15.3:0/1160782986 >> v2:172.21.15.203:3300/0 conn(0x7f0d780a6d10 0x7f0d781476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7d408640 1 -- 172.21.15.3:0/1160782986 >> v2:172.21.15.203:3300/0 conn(0x7f0d780a6d10 msgr2=0x7f0d781476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7d408640 1 --2- 172.21.15.3:0/1160782986 >> v2:172.21.15.203:3300/0 conn(0x7f0d780a6d10 0x7f0d781476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7d408640 1 -- 172.21.15.3:0/1160782986 >> v1:172.21.15.3:6789/0 conn(0x7f0d780a7180 legacy=0x7f0d78147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.307+0000 7f0d7d408640 1 -- 172.21.15.3:0/1160782986 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d78145090 con 0x7f0d78144cb0 2024-05-21T17:41:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.308+0000 7f0d7d408640 1 --2- 172.21.15.3:0/1160782986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d78144cb0 0x7f0d78073e40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d78071db0 tx=0x7f0d7402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82140cf85c70f8a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.308+0000 7f0d6ffff640 1 -- 172.21.15.3:0/1160782986 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74039070 con 0x7f0d78144cb0 2024-05-21T17:41:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.308+0000 7f0d6ffff640 1 -- 172.21.15.3:0/1160782986 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d7402cbf0 con 0x7f0d78144cb0 2024-05-21T17:41:18.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.308+0000 7f0d6ffff640 1 -- 172.21.15.3:0/1160782986 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74034440 con 0x7f0d78144cb0 2024-05-21T17:41:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.308+0000 7f0d7f66c640 1 -- 172.21.15.3:0/1160782986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d78144cb0 msgr2=0x7f0d78073e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.308+0000 7f0d7f66c640 1 --2- 172.21.15.3:0/1160782986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d78144cb0 0x7f0d78073e40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d78071db0 tx=0x7f0d7402be40 comp rx=0 tx=0).stop 2024-05-21T17:41:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.309+0000 7f0d7f66c640 1 -- 172.21.15.3:0/1160782986 shutdown_connections 2024-05-21T17:41:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.309+0000 7f0d7f66c640 1 --2- 172.21.15.3:0/1160782986 >> v2:172.21.15.203:3300/0 conn(0x7f0d780a6d10 0x7f0d781476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.309+0000 7f0d7f66c640 1 --2- 172.21.15.3:0/1160782986 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d78144cb0 0x7f0d78073e40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.309+0000 7f0d7f66c640 1 -- 172.21.15.3:0/1160782986 >> 172.21.15.3:0/1160782986 conn(0x7f0d780a8c40 msgr2=0x7f0d780a9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.309+0000 7f0d7f66c640 1 -- 172.21.15.3:0/1160782986 shutdown_connections 2024-05-21T17:41:18.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.309+0000 7f0d7f66c640 1 -- 172.21.15.3:0/1160782986 wait complete. 2024-05-21T17:41:18.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7f66c640 1 Processor -- start 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7f66c640 1 -- start start 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7f66c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d780a6d10 0x7f0d7809ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7f66c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d780a7180 0x7f0d781d9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7f66c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d78144cb0 0x7f0d781dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7f66c640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0d780739c0 con 0x7f0d780a7180 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7f66c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78144360 con 0x7f0d780a6d10 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7f66c640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f0d780749c0 con 0x7f0d78144cb0 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7cc07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d780a7180 0x7f0d781d9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7dc09640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d78144cb0 0x7f0d781dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.310+0000 7f0d7cc07640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d780a7180 0x7f0d781d9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50014/0 (socket says 172.21.15.3:50014) 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d7cc07640 1 -- 172.21.15.3:0/2477125539 learned_addr learned my addr 172.21.15.3:0/2477125539 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d7d408640 1 --2- 172.21.15.3:0/2477125539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d780a6d10 0x7f0d7809ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d7cc07640 1 -- 172.21.15.3:0/2477125539 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d78144cb0 msgr2=0x7f0d781dd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d7cc07640 1 --2- 172.21.15.3:0/2477125539 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d78144cb0 0x7f0d781dd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d7cc07640 1 -- 172.21.15.3:0/2477125539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d780a6d10 msgr2=0x7f0d7809ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d7cc07640 1 --2- 172.21.15.3:0/2477125539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d780a6d10 0x7f0d7809ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d7cc07640 1 -- 172.21.15.3:0/2477125539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d781da740 con 0x7f0d780a7180 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d7cc07640 1 --2- 172.21.15.3:0/2477125539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d780a7180 0x7f0d781d9b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800d800 tx=0x7f0d6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d6dffb640 1 -- 172.21.15.3:0/2477125539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800b6c0 con 0x7f0d780a7180 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d6dffb640 1 -- 172.21.15.3:0/2477125539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d68019070 con 0x7f0d780a7180 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d7f66c640 1 -- 172.21.15.3:0/2477125539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d781d0960 con 0x7f0d780a7180 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d6dffb640 1 -- 172.21.15.3:0/2477125539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800ba80 con 0x7f0d780a7180 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.311+0000 7f0d7f66c640 1 -- 172.21.15.3:0/2477125539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d781d0600 con 0x7f0d780a7180 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.312+0000 7f0d6dffb640 1 -- 172.21.15.3:0/2477125539 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d6800bc20 con 0x7f0d780a7180 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.312+0000 7f0d6dffb640 20 mgrc handle_mgr_map mgrmap(e 1) 2024-05-21T17:41:18.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.312+0000 7f0d6dffb640 4 mgrc handle_mgr_map Got map version 1 2024-05-21T17:41:18.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.312+0000 7f0d6dffb640 4 mgrc handle_mgr_map Active mgr is now 2024-05-21T17:41:18.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.312+0000 7f0d6dffb640 4 mgrc reconnect No active mgr available yet 2024-05-21T17:41:18.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.312+0000 7f0d6dffb640 1 -- 172.21.15.3:0/2477125539 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f0d68014a30 con 0x7f0d780a7180 2024-05-21T17:41:18.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.313+0000 7f0d7f66c640 1 -- 172.21.15.3:0/2477125539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d78069290 con 0x7f0d780a7180 2024-05-21T17:41:18.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.315+0000 7f0d6dffb640 1 -- 172.21.15.3:0/2477125539 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0d68014d70 con 0x7f0d780a7180 2024-05-21T17:41:18.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.378+0000 7f0d7f66c640 1 -- 172.21.15.3:0/2477125539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d78073c20 con 0x7f0d780a7180 2024-05-21T17:41:18.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.378+0000 7f0d6dffb640 1 -- 172.21.15.3:0/2477125539 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) ==== 74+0+4153 (secure 0 0 0) 0x7f0d68014520 con 0x7f0d780a7180 2024-05-21T17:41:18.380 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:18.380 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"fsid":"e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442","created":"2024-05-21T17:41:17.034337+0000","modified":"2024-05-21T17:41:18.179591+0000","last_up_change":"0.000000","last_in_change":"2024-05-21T17:41:18.179591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a7ac7071-219d-4cf4-9f8a-977f495fd0d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"89df5105-5900-4ffa-b8c1-6f480865a085","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"15f83334-8efc-40c2-af0c-eeed45923086","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6a27ac8e-f957-4455-b477-3090081766c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T17:41:18.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.379+0000 7f0d7f66c640 10 mgrc shutdown 2024-05-21T17:41:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.379+0000 7f0d7f66c640 1 -- 172.21.15.3:0/2477125539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d780a7180 msgr2=0x7f0d781d9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.379+0000 7f0d7f66c640 1 --2- 172.21.15.3:0/2477125539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d780a7180 0x7f0d781d9b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800d800 tx=0x7f0d6800dcc0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.379+0000 7f0d7f66c640 1 -- 172.21.15.3:0/2477125539 shutdown_connections 2024-05-21T17:41:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.379+0000 7f0d7f66c640 1 --2- 172.21.15.3:0/2477125539 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0d78144cb0 0x7f0d781dd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.379+0000 7f0d7f66c640 1 --2- 172.21.15.3:0/2477125539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0d780a7180 0x7f0d781d9b30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.379+0000 7f0d7f66c640 1 --2- 172.21.15.3:0/2477125539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d780a6d10 0x7f0d7809ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.379+0000 7f0d7f66c640 1 -- 172.21.15.3:0/2477125539 >> 172.21.15.3:0/2477125539 conn(0x7f0d780a8c40 msgr2=0x7f0d780b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.379+0000 7f0d7f66c640 1 -- 172.21.15.3:0/2477125539 shutdown_connections 2024-05-21T17:41:18.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:18.379+0000 7f0d7f66c640 1 -- 172.21.15.3:0/2477125539 wait complete. 2024-05-21T17:41:18.392 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-21T17:41:18.392 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-21T17:41:18.495 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:18.494+0000 7ff3a7c0f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T17:41:18.495 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:18.494+0000 7f3819eda040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T17:41:18.526 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:18.524+0000 7f8bf81b8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T17:41:18.587 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:18.586+0000 7ff3a7c0f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T17:41:18.588 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:18.586+0000 7f3819eda040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T17:41:18.617 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:18.615+0000 7f8bf81b8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T17:41:18.632 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:18.631+0000 7ff3a7c0f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T17:41:18.632 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:18.631+0000 7f3819eda040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T17:41:18.662 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:18.660+0000 7f8bf81b8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T17:41:18.791 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-05-21T17:41:18.789+0000 7fd07be0d640 -1 Falling back to public interface 2024-05-21T17:41:18.797 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-21T17:41:18.799 INFO:tasks.ceph.osd.1.smithi203.stderr:2024-05-21T17:41:18.799+0000 7f04d52d9640 -1 Falling back to public interface 2024-05-21T17:41:18.800 INFO:tasks.ceph.osd.3.smithi203.stderr:2024-05-21T17:41:18.799+0000 7fd77521f640 -1 Falling back to public interface 2024-05-21T17:41:18.808 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:18.807+0000 7f3819eda040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T17:41:18.808 INFO:tasks.ceph.osd.2.smithi203.stderr:2024-05-21T17:41:18.807+0000 7f2ec8516640 -1 Falling back to public interface 2024-05-21T17:41:18.809 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:18.808+0000 7ff3a7c0f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T17:41:18.838 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:18.836+0000 7f8bf81b8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T17:41:18.869 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:18.868+0000 7ff3a7c0f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T17:41:18.870 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:18.868+0000 7f3819eda040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T17:41:18.891 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.889+0000 7ff09c94a640 1 Processor -- start 2024-05-21T17:41:18.891 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.889+0000 7ff09c94a640 1 -- start start 2024-05-21T17:41:18.892 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff09c94a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff0981458f0 0x7ff098145d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.892 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff09c94a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09814eaf0 0x7ff098150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.892 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff09c94a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff098070270 con 0x7ff0980a7040 2024-05-21T17:41:18.892 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff09c94a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff098070a60 con 0x7ff0981458f0 2024-05-21T17:41:18.893 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff09c94a640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7ff0980a8510 con 0x7ff09814eaf0 2024-05-21T17:41:18.893 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff096d9d640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09814eaf0 0x7ff098150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.893 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff095d9b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff0981458f0 0x7ff098145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.893 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff09659c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff0980a7040 0x7ff0980a9270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:33200/0 (socket says 172.21.15.3:33200) 2024-05-21T17:41:18.893 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff09659c640 1 -- 172.21.15.3:0/2951310243 learned_addr learned my addr 172.21.15.3:0/2951310243 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:18.893 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff095d9b640 1 -- 172.21.15.3:0/2951310243 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09814eaf0 msgr2=0x7ff098150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:18.893 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff095d9b640 1 --2- 172.21.15.3:0/2951310243 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09814eaf0 0x7ff098150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.893 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff095d9b640 1 -- 172.21.15.3:0/2951310243 >> v1:172.21.15.3:6789/0 conn(0x7ff0980a7040 legacy=0x7ff0980a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.894 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff09559a640 1 -- 172.21.15.3:0/2951310243 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3328910770 0 0) 0x7ff098070270 con 0x7ff0980a7040 2024-05-21T17:41:18.894 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.890+0000 7ff095d9b640 1 -- 172.21.15.3:0/2951310243 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0980a87d0 con 0x7ff0981458f0 2024-05-21T17:41:18.894 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:18.891+0000 7f8bf81b8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T17:41:18.898 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.891+0000 7ff095d9b640 1 --2- 172.21.15.3:0/2951310243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0981458f0 0x7ff098145d10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff0800097a0 tx=0x7ff08002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a395047a8a7d26e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:18.898 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.891+0000 7ff09559a640 1 -- 172.21.15.3:0/2951310243 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0800315a0 con 0x7ff0981458f0 2024-05-21T17:41:18.898 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.891+0000 7ff09559a640 1 -- 172.21.15.3:0/2951310243 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff080037070 con 0x7ff0981458f0 2024-05-21T17:41:18.898 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.891+0000 7ff09559a640 1 -- 172.21.15.3:0/2951310243 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080031960 con 0x7ff0981458f0 2024-05-21T17:41:18.898 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.892+0000 7ff09c94a640 1 -- 172.21.15.3:0/2951310243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0981458f0 msgr2=0x7ff098145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.898 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.892+0000 7ff09c94a640 1 --2- 172.21.15.3:0/2951310243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0981458f0 0x7ff098145d10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff0800097a0 tx=0x7ff08002b840 comp rx=0 tx=0).stop 2024-05-21T17:41:18.898 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.892+0000 7ff09c94a640 1 -- 172.21.15.3:0/2951310243 shutdown_connections 2024-05-21T17:41:18.898 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.892+0000 7ff09c94a640 1 --2- 172.21.15.3:0/2951310243 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff09814eaf0 0x7ff098150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.892+0000 7ff09c94a640 1 --2- 172.21.15.3:0/2951310243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0981458f0 0x7ff098145d10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.892+0000 7ff09c94a640 1 -- 172.21.15.3:0/2951310243 >> 172.21.15.3:0/2951310243 conn(0x7ff09806c9a0 msgr2=0x7ff0980a8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.892+0000 7ff09c94a640 1 -- 172.21.15.3:0/2951310243 shutdown_connections 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.892+0000 7ff09c94a640 1 -- 172.21.15.3:0/2951310243 wait complete. 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.893+0000 7ff09c94a640 1 Processor -- start 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.893+0000 7ff09c94a640 1 -- start start 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff09c94a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0980a7040 0x7ff09813ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff09c94a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0981458f0 0x7ff09813d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff09c94a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff09814eaf0 0x7ff09813d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff09c94a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff0980a7420 con 0x7ff0980a7040 2024-05-21T17:41:18.899 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff09c94a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff098071370 con 0x7ff09814eaf0 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff09c94a640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7ff0980a8190 con 0x7ff0981458f0 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff09659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0980a7040 0x7ff09813ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff095d9b640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0981458f0 0x7ff09813d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff095d9b640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0981458f0 0x7ff09813d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41534/0 (socket says 172.21.15.3:41534) 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff095d9b640 1 -- 172.21.15.3:0/3396622196 learned_addr learned my addr 172.21.15.3:0/3396622196 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff096d9d640 1 --2- 172.21.15.3:0/3396622196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff09814eaf0 0x7ff09813d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff095d9b640 1 -- 172.21.15.3:0/3396622196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff09814eaf0 msgr2=0x7ff09813d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff095d9b640 1 --2- 172.21.15.3:0/3396622196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff09814eaf0 0x7ff09813d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff095d9b640 1 -- 172.21.15.3:0/3396622196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0980a7040 msgr2=0x7ff09813ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff095d9b640 1 --2- 172.21.15.3:0/3396622196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0980a7040 0x7ff09813ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.900 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff095d9b640 1 -- 172.21.15.3:0/3396622196 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09813dde0 con 0x7ff0981458f0 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff09659c640 1 --2- 172.21.15.3:0/3396622196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0980a7040 0x7ff09813ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.894+0000 7ff095d9b640 1 --2- 172.21.15.3:0/3396622196 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0981458f0 0x7ff09813d150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff080034040 tx=0x7ff0800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff08effd640 1 -- 172.21.15.3:0/3396622196 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080032630 con 0x7ff0981458f0 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff08effd640 1 -- 172.21.15.3:0/3396622196 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0800327f0 con 0x7ff0981458f0 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff09c94a640 1 -- 172.21.15.3:0/3396622196 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09813e0a0 con 0x7ff0981458f0 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff08effd640 1 -- 172.21.15.3:0/3396622196 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080032be0 con 0x7ff0981458f0 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff09c94a640 1 -- 172.21.15.3:0/3396622196 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09813f3a0 con 0x7ff0981458f0 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff08effd640 1 -- 172.21.15.3:0/3396622196 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff080006020 con 0x7ff0981458f0 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff08effd640 20 mgrc handle_mgr_map mgrmap(e 1) 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff08effd640 4 mgrc handle_mgr_map Got map version 1 2024-05-21T17:41:18.901 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff08effd640 4 mgrc handle_mgr_map Active mgr is now 2024-05-21T17:41:18.902 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff08effd640 4 mgrc reconnect No active mgr available yet 2024-05-21T17:41:18.902 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.895+0000 7ff08effd640 1 -- 172.21.15.3:0/3396622196 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7ff08002f030 con 0x7ff0981458f0 2024-05-21T17:41:18.902 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.896+0000 7ff09c94a640 1 -- 172.21.15.3:0/3396622196 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0980696d0 con 0x7ff0981458f0 2024-05-21T17:41:18.902 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.898+0000 7ff08effd640 1 -- 172.21.15.3:0/3396622196 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff080040070 con 0x7ff0981458f0 2024-05-21T17:41:18.924 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:18.923+0000 7ff3a7c0f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T17:41:18.925 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:18.924+0000 7f3819eda040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T17:41:18.949 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:18.947+0000 7f8bf81b8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T17:41:18.962 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.960+0000 7ff09c94a640 1 -- 172.21.15.3:0/3396622196 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0980704d0 con 0x7ff0981458f0 2024-05-21T17:41:18.963 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.961+0000 7ff08effd640 1 -- 172.21.15.3:0/3396622196 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) ==== 74+0+4153 (secure 0 0 0) 0x7ff080032d80 con 0x7ff0981458f0 2024-05-21T17:41:18.963 INFO:teuthology.misc.health.smithi003.stdout: 2024-05-21T17:41:18.963 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":6,"fsid":"e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442","created":"2024-05-21T17:41:17.034337+0000","modified":"2024-05-21T17:41:18.179591+0000","last_up_change":"0.000000","last_in_change":"2024-05-21T17:41:18.179591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a7ac7071-219d-4cf4-9f8a-977f495fd0d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"89df5105-5900-4ffa-b8c1-6f480865a085","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"15f83334-8efc-40c2-af0c-eeed45923086","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6a27ac8e-f957-4455-b477-3090081766c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T17:41:18.964 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.961+0000 7ff09c94a640 10 mgrc shutdown 2024-05-21T17:41:18.964 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.962+0000 7ff09c94a640 1 -- 172.21.15.3:0/3396622196 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0981458f0 msgr2=0x7ff09813d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:18.964 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.962+0000 7ff09c94a640 1 --2- 172.21.15.3:0/3396622196 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0981458f0 0x7ff09813d150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff080034040 tx=0x7ff0800046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.964 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.962+0000 7ff09c94a640 1 -- 172.21.15.3:0/3396622196 shutdown_connections 2024-05-21T17:41:18.964 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.962+0000 7ff09c94a640 1 --2- 172.21.15.3:0/3396622196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff09814eaf0 0x7ff09813d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.964 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.962+0000 7ff09c94a640 1 --2- 172.21.15.3:0/3396622196 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff0981458f0 0x7ff09813d150 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.964 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.962+0000 7ff09c94a640 1 --2- 172.21.15.3:0/3396622196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0980a7040 0x7ff09813ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:18.965 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.962+0000 7ff09c94a640 1 -- 172.21.15.3:0/3396622196 >> 172.21.15.3:0/3396622196 conn(0x7ff09806c9a0 msgr2=0x7ff098147b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:18.965 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.962+0000 7ff09c94a640 1 -- 172.21.15.3:0/3396622196 shutdown_connections 2024-05-21T17:41:18.965 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:18.962+0000 7ff09c94a640 1 -- 172.21.15.3:0/3396622196 wait complete. 2024-05-21T17:41:18.973 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-05-21T17:41:19.042 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:19.041+0000 7ff3a7c0f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:41:19.043 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:19.042+0000 7f3819eda040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:41:19.068 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:19.066+0000 7f8bf81b8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:41:19.447 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:19.446+0000 7ff3a7c0f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T17:41:19.448 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:19.447+0000 7f3819eda040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T17:41:19.467 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:19.466+0000 7f8bf81b8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T17:41:19.594 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:19.593+0000 7f3819eda040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T17:41:19.594 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:19.593+0000 7ff3a7c0f040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T17:41:19.613 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:19.611+0000 7f8bf81b8040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T17:41:19.869 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:19.867+0000 7ff3a7c0f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T17:41:19.869 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:19.867+0000 7f3819eda040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T17:41:19.885 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:19.883+0000 7f8bf81b8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T17:41:19.924 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:19.923+0000 7ff3a7c0f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T17:41:19.924 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:19.923+0000 7f3819eda040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T17:41:19.940 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:19.938+0000 7f8bf81b8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T17:41:20.046 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:20.045+0000 7ff3a7c0f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T17:41:20.047 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:20.046+0000 7f3819eda040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T17:41:20.061 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:20.059+0000 7f8bf81b8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T17:41:20.349 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:20.347+0000 7f3819eda040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T17:41:20.349 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:20.347+0000 7ff3a7c0f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T17:41:20.362 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:20.360+0000 7f8bf81b8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T17:41:20.399 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:20.398+0000 7f3819eda040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T17:41:20.400 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:20.398+0000 7ff3a7c0f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T17:41:20.412 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:20.410+0000 7f8bf81b8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T17:41:20.565 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:20.564+0000 7ff3a7c0f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T17:41:20.565 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:20.564+0000 7f3819eda040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T17:41:20.576 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:20.574+0000 7f8bf81b8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T17:41:20.708 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:20.707+0000 7f3819eda040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T17:41:20.709 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:20.708+0000 7ff3a7c0f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T17:41:20.720 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:20.719+0000 7f8bf81b8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T17:41:20.761 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:20.760+0000 7f3819eda040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T17:41:20.761 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:20.760+0000 7ff3a7c0f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T17:41:20.772 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:20.771+0000 7f8bf81b8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T17:41:20.835 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:20.833+0000 7ff3a7c0f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T17:41:20.835 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:20.834+0000 7f3819eda040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T17:41:20.846 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:20.844+0000 7f8bf81b8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T17:41:20.882 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:20.881+0000 7ff3a7c0f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T17:41:20.883 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:20.882+0000 7f3819eda040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T17:41:20.893 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:20.892+0000 7f8bf81b8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T17:41:21.388 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:21.386+0000 7ff3a7c0f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T17:41:21.388 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:21.387+0000 7f3819eda040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T17:41:21.392 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:21.390+0000 7f8bf81b8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T17:41:21.509 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:21.508+0000 7ff3a7c0f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T17:41:21.510 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:41:21.509+0000 7f3819eda040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T17:41:21.515 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:41:21.513+0000 7f8bf81b8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T17:41:22.272 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-05-21T17:41:22.270+0000 7fd07be0d640 -1 osd.0 0 log_to_monitors true 2024-05-21T17:41:22.278 INFO:tasks.ceph.osd.2.smithi203.stderr:2024-05-21T17:41:22.277+0000 7f2ec8516640 -1 osd.2 0 log_to_monitors true 2024-05-21T17:41:22.300 INFO:tasks.ceph.osd.3.smithi203.stderr:2024-05-21T17:41:22.299+0000 7fd77521f640 -1 osd.3 0 log_to_monitors true 2024-05-21T17:41:22.317 INFO:tasks.ceph.osd.1.smithi203.stderr:2024-05-21T17:41:22.317+0000 7f04d52d9640 -1 osd.1 0 log_to_monitors true 2024-05-21T17:41:22.526 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:22.525+0000 7ff3473d8640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-21T17:41:22.526 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:41:22.525+0000 7ff3473d8640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-21T17:41:24.135 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-05-21T17:41:24.133+0000 7fd0777aa640 -1 osd.0 0 waiting for initial osdmap 2024-05-21T17:41:24.150 INFO:tasks.ceph.osd.0.smithi003.stderr:2024-05-21T17:41:24.148+0000 7fd06f5ca640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T17:41:24.448 INFO:tasks.ceph.osd.1.smithi203.stderr:2024-05-21T17:41:24.446+0000 7f04d126f640 -1 osd.1 0 waiting for initial osdmap 2024-05-21T17:41:24.458 INFO:tasks.ceph.osd.1.smithi203.stderr:2024-05-21T17:41:24.457+0000 7f04c908f640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T17:41:24.469 INFO:tasks.ceph.osd.3.smithi203.stderr:2024-05-21T17:41:24.468+0000 7fd770c64640 -1 osd.3 0 waiting for initial osdmap 2024-05-21T17:41:24.479 INFO:tasks.ceph.osd.2.smithi203.stderr:2024-05-21T17:41:24.478+0000 7f2ec44ac640 -1 osd.2 0 waiting for initial osdmap 2024-05-21T17:41:24.481 INFO:tasks.ceph.osd.3.smithi203.stderr:2024-05-21T17:41:24.480+0000 7fd768a84640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T17:41:24.498 INFO:tasks.ceph.osd.2.smithi203.stderr:2024-05-21T17:41:24.497+0000 7f2ebc2cc640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-21T17:41:25.378 DEBUG:teuthology.orchestra.run.smithi003:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-21T17:41:25.484 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.482+0000 7ff460579640 1 Processor -- start 2024-05-21T17:41:25.484 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.482+0000 7ff460579640 1 -- start start 2024-05-21T17:41:25.486 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.483+0000 7ff460579640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4580b8ba0 0x7ff458142650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.486 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.483+0000 7ff460579640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4580aa220 0x7ff45814d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.486 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.483+0000 7ff460579640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff458143600 con 0x7ff4580aaa40 2024-05-21T17:41:25.486 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.483+0000 7ff460579640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff4580a9390 con 0x7ff4580aa220 2024-05-21T17:41:25.486 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.483+0000 7ff460579640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff4581438c0 con 0x7ff4580b8ba0 2024-05-21T17:41:25.486 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.483+0000 7ff45eb16640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4580aa220 0x7ff45814d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.486 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.483+0000 7ff45eb16640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff4580aa220 0x7ff45814d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50172/0 (socket says 172.21.15.3:50172) 2024-05-21T17:41:25.486 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.483+0000 7ff45db14640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff4580aaa40 0x7ff458142b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35720/0 (socket says 172.21.15.3:35720) 2024-05-21T17:41:25.486 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.483+0000 7ff45eb16640 1 -- 172.21.15.3:0/1394450053 learned_addr learned my addr 172.21.15.3:0/1394450053 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:25.487 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.483+0000 7ff45e315640 1 --2- 172.21.15.3:0/1394450053 >> v2:172.21.15.203:3300/0 conn(0x7ff4580b8ba0 0x7ff458142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.487 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.484+0000 7ff45eb16640 1 -- 172.21.15.3:0/1394450053 >> v2:172.21.15.203:3300/0 conn(0x7ff4580b8ba0 msgr2=0x7ff458142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:25.487 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.484+0000 7ff45eb16640 1 --2- 172.21.15.3:0/1394450053 >> v2:172.21.15.203:3300/0 conn(0x7ff4580b8ba0 0x7ff458142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.487 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.484+0000 7ff45eb16640 1 -- 172.21.15.3:0/1394450053 >> v1:172.21.15.3:6789/0 conn(0x7ff4580aaa40 legacy=0x7ff458142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:25.487 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.484+0000 7ff45d313640 1 -- 172.21.15.3:0/1394450053 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2085009166 0 0) 0x7ff458143600 con 0x7ff4580aaa40 2024-05-21T17:41:25.487 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.484+0000 7ff45eb16640 1 -- 172.21.15.3:0/1394450053 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4580a5950 con 0x7ff4580aa220 2024-05-21T17:41:25.488 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.484+0000 7ff45eb16640 1 --2- 172.21.15.3:0/1394450053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4580aa220 0x7ff45814d7a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4580b9a20 tx=0x7ff45002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8964e5e259081e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:25.488 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff45d313640 1 -- 172.21.15.3:0/1394450053 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450034070 con 0x7ff4580aa220 2024-05-21T17:41:25.488 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff45d313640 1 -- 172.21.15.3:0/1394450053 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff450030860 con 0x7ff4580aa220 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff45d313640 1 -- 172.21.15.3:0/1394450053 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450030c30 con 0x7ff4580aa220 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff460579640 1 -- 172.21.15.3:0/1394450053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4580aa220 msgr2=0x7ff45814d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff460579640 1 --2- 172.21.15.3:0/1394450053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4580aa220 0x7ff45814d7a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4580b9a20 tx=0x7ff45002b9a0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff460579640 1 -- 172.21.15.3:0/1394450053 shutdown_connections 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff460579640 1 --2- 172.21.15.3:0/1394450053 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff4580aa220 0x7ff45814d7a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff460579640 1 --2- 172.21.15.3:0/1394450053 >> v2:172.21.15.203:3300/0 conn(0x7ff4580b8ba0 0x7ff458142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff460579640 1 -- 172.21.15.3:0/1394450053 >> 172.21.15.3:0/1394450053 conn(0x7ff458071550 msgr2=0x7ff45806d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff460579640 1 -- 172.21.15.3:0/1394450053 shutdown_connections 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.485+0000 7ff460579640 1 -- 172.21.15.3:0/1394450053 wait complete. 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.486+0000 7ff460579640 1 Processor -- start 2024-05-21T17:41:25.489 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.486+0000 7ff460579640 1 -- start start 2024-05-21T17:41:25.491 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff460579640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4580aa220 0x7ff458139ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.491 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff460579640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4580b8ba0 0x7ff458145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.492 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff460579640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff458145760 0x7ff4580b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.492 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff460579640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff4580a9cf0 con 0x7ff458145760 2024-05-21T17:41:25.492 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff460579640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4580a9b70 con 0x7ff4580aa220 2024-05-21T17:41:25.492 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff460579640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7ff45814dce0 con 0x7ff4580b8ba0 2024-05-21T17:41:25.492 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45eb16640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff458145760 0x7ff4580b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.492 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45eb16640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff458145760 0x7ff4580b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50176/0 (socket says 172.21.15.3:50176) 2024-05-21T17:41:25.492 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45db14640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4580b8ba0 0x7ff458145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.492 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45eb16640 1 -- 172.21.15.3:0/1726404086 learned_addr learned my addr 172.21.15.3:0/1726404086 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:25.492 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45e315640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4580aa220 0x7ff458139ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45eb16640 1 -- 172.21.15.3:0/1726404086 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4580b8ba0 msgr2=0x7ff458145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45eb16640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4580b8ba0 0x7ff458145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45eb16640 1 -- 172.21.15.3:0/1726404086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4580aa220 msgr2=0x7ff458139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45eb16640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4580aa220 0x7ff458139ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45eb16640 1 -- 172.21.15.3:0/1726404086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4580b4560 con 0x7ff458145760 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.487+0000 7ff45db14640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4580b8ba0 0x7ff458145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.488+0000 7ff45eb16640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff458145760 0x7ff4580b4bf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff45002b9e0 tx=0x7ff450004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.488+0000 7ff44effd640 1 -- 172.21.15.3:0/1726404086 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450048070 con 0x7ff458145760 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.488+0000 7ff44effd640 1 -- 172.21.15.3:0/1726404086 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff450030660 con 0x7ff458145760 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.488+0000 7ff460579640 1 -- 172.21.15.3:0/1726404086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4580b58c0 con 0x7ff458145760 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.488+0000 7ff44effd640 1 -- 172.21.15.3:0/1726404086 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450034040 con 0x7ff458145760 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.488+0000 7ff460579640 1 -- 172.21.15.3:0/1726404086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff458139150 con 0x7ff458145760 2024-05-21T17:41:25.493 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.489+0000 7ff460579640 1 -- 172.21.15.3:0/1726404086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4580696d0 con 0x7ff458145760 2024-05-21T17:41:25.494 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.490+0000 7ff44effd640 1 -- 172.21.15.3:0/1726404086 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 146935+0+0 (secure 0 0 0) 0x7ff45004b070 con 0x7ff458145760 2024-05-21T17:41:25.494 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.490+0000 7ff44effd640 20 mgrc handle_mgr_map mgrmap(e 4) 2024-05-21T17:41:25.494 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.491+0000 7ff44effd640 4 mgrc handle_mgr_map Got map version 4 2024-05-21T17:41:25.495 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.491+0000 7ff44effd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:25.495 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.491+0000 7ff44effd640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:25.495 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.491+0000 7ff44effd640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff43c0af1d0 0x7ff43c0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.495 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.491+0000 7ff44effd640 1 -- 172.21.15.3:0/1726404086 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 2743+0+0 (secure 0 0 0) 0x7ff4500fe2e0 con 0x7ff458145760 2024-05-21T17:41:25.495 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.492+0000 7ff45e315640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff43c0af1d0 0x7ff43c0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.495 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.492+0000 7ff45e315640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff43c0af1d0 0x7ff43c0b1690 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4540099c0 tx=0x7ff454003990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:25.498 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.496+0000 7ff44effd640 1 -- 172.21.15.3:0/1726404086 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff450080990 con 0x7ff458145760 2024-05-21T17:41:25.695 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.693+0000 7ff460579640 1 -- 172.21.15.3:0/1726404086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff458145fc0 con 0x7ff458145760 2024-05-21T17:41:25.696 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.694+0000 7ff44effd640 1 -- 172.21.15.3:0/1726404086 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) ==== 74+0+6092 (secure 0 0 0) 0x7ff4500e9090 con 0x7ff458145760 2024-05-21T17:41:25.696 INFO:teuthology.misc.health.smithi003.stdout: 2024-05-21T17:41:25.696 INFO:teuthology.misc.health.smithi003.stdout:{"epoch":9,"fsid":"e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442","created":"2024-05-21T17:41:17.034337+0000","modified":"2024-05-21T17:41:25.136644+0000","last_up_change":"2024-05-21T17:41:25.136644+0000","last_in_change":"2024-05-21T17:41:18.179591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a7ac7071-219d-4cf4-9f8a-977f495fd0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6801","nonce":3296405141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6803","nonce":3296405141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6807","nonce":3296405141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6805","nonce":3296405141}]},"public_addr":"172.21.15.3:6801/3296405141","cluster_addr":"172.21.15.3:6803/3296405141","heartbeat_back_addr":"172.21.15.3:6807/3296405141","heartbeat_front_addr":"172.21.15.3:6805/3296405141","state":["exists","up"]},{"osd":1,"uuid":"89df5105-5900-4ffa-b8c1-6f480865a085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6801","nonce":1777872358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6803","nonce":1777872358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6810","nonce":1777872358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6805","nonce":1777872358}]},"public_addr":"172.21.15.203:6801/1777872358","cluster_addr":"172.21.15.203:6803/1777872358","heartbeat_back_addr":"172.21.15.203:6810/1777872358","heartbeat_front_addr":"172.21.15.203:6805/1777872358","state":["exists","up"]},{"osd":2,"uuid":"15f83334-8efc-40c2-af0c-eeed45923086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6817","nonce":1069390381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6819","nonce":1069390381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6823","nonce":1069390381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6821","nonce":1069390381}]},"public_addr":"172.21.15.203:6817/1069390381","cluster_addr":"172.21.15.203:6819/1069390381","heartbeat_back_addr":"172.21.15.203:6823/1069390381","heartbeat_front_addr":"172.21.15.203:6821/1069390381","state":["exists","up"]},{"osd":3,"uuid":"6a27ac8e-f957-4455-b477-3090081766c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6808","nonce":1072938612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6811","nonce":1072938612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6815","nonce":1072938612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6813","nonce":1072938612}]},"public_addr":"172.21.15.203:6808/1072938612","cluster_addr":"172.21.15.203:6811/1072938612","heartbeat_back_addr":"172.21.15.203:6815/1072938612","heartbeat_front_addr":"172.21.15.203:6813/1072938612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T17:41:25.698 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 10 mgrc shutdown 2024-05-21T17:41:25.698 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 -- 172.21.15.3:0/1726404086 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff43c0af1d0 msgr2=0x7ff43c0b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:25.698 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff43c0af1d0 0x7ff43c0b1690 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4540099c0 tx=0x7ff454003990 comp rx=0 tx=0).stop 2024-05-21T17:41:25.698 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 -- 172.21.15.3:0/1726404086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff458145760 msgr2=0x7ff4580b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:25.698 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff458145760 0x7ff4580b4bf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff45002b9e0 tx=0x7ff450004730 comp rx=0 tx=0).stop 2024-05-21T17:41:25.699 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 -- 172.21.15.3:0/1726404086 shutdown_connections 2024-05-21T17:41:25.699 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff43c0af1d0 0x7ff43c0b1690 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.699 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff458145760 0x7ff4580b4bf0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.699 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff4580b8ba0 0x7ff458145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.699 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 --2- 172.21.15.3:0/1726404086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4580aa220 0x7ff458139ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.699 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 -- 172.21.15.3:0/1726404086 >> 172.21.15.3:0/1726404086 conn(0x7ff458071550 msgr2=0x7ff45806e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:25.699 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 -- 172.21.15.3:0/1726404086 shutdown_connections 2024-05-21T17:41:25.699 INFO:teuthology.misc.health.smithi003.stderr:2024-05-21T17:41:25.696+0000 7ff460579640 1 -- 172.21.15.3:0/1726404086 wait complete. 2024-05-21T17:41:25.708 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-05-21T17:41:25.708 INFO:tasks.ceph:Creating RBD pool 2024-05-21T17:41:25.708 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-21T17:41:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.809+0000 7fb14d0d4640 1 Processor -- start 2024-05-21T17:41:25.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.809+0000 7fb14d0d4640 1 -- start start 2024-05-21T17:41:25.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.809+0000 7fb14d0d4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1480ac300 0x7fb148149680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.809+0000 7fb14d0d4640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb1480adbd0 0x7fb148150a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.809+0000 7fb14d0d4640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb148146640 con 0x7fb14806cc60 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.809+0000 7fb14d0d4640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb148146b80 con 0x7fb1480adbd0 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.809+0000 7fb14d0d4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb1480ac6e0 con 0x7fb1480ac300 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.809+0000 7fb14759e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb1480adbd0 0x7fb148150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.810+0000 7fb14759e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb1480adbd0 0x7fb148150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50180/0 (socket says 172.21.15.3:50180) 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.810+0000 7fb14759e640 1 -- 172.21.15.3:0/535151914 learned_addr learned my addr 172.21.15.3:0/535151914 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.810+0000 7fb146d9d640 1 --2- 172.21.15.3:0/535151914 >> v2:172.21.15.203:3300/0 conn(0x7fb1480ac300 0x7fb148149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.810+0000 7fb14759e640 1 -- 172.21.15.3:0/535151914 >> v2:172.21.15.203:3300/0 conn(0x7fb1480ac300 msgr2=0x7fb148149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.810+0000 7fb14759e640 1 --2- 172.21.15.3:0/535151914 >> v2:172.21.15.203:3300/0 conn(0x7fb1480ac300 0x7fb148149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.810+0000 7fb14759e640 1 -- 172.21.15.3:0/535151914 >> v1:172.21.15.3:6789/0 conn(0x7fb14806cc60 legacy=0x7fb148149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.810+0000 7fb14759e640 1 -- 172.21.15.3:0/535151914 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1480a8300 con 0x7fb1480adbd0 2024-05-21T17:41:25.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.810+0000 7fb145d9b640 1 -- 172.21.15.3:0/535151914 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 830633175 0 0) 0x7fb148146640 con 0x7fb14806cc60 2024-05-21T17:41:25.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.810+0000 7fb146d9d640 1 --2- 172.21.15.3:0/535151914 >> v2:172.21.15.203:3300/0 conn(0x7fb1480ac300 0x7fb148149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:25.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb14759e640 1 --2- 172.21.15.3:0/535151914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480adbd0 0x7fb148150a10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb1480bca80 tx=0x7fb14002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c29ee1f18fa58be server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:25.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb145d9b640 1 -- 172.21.15.3:0/535151914 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14002c9f0 con 0x7fb1480adbd0 2024-05-21T17:41:25.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb145d9b640 1 -- 172.21.15.3:0/535151914 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb14002cb90 con 0x7fb1480adbd0 2024-05-21T17:41:25.814 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb145d9b640 1 -- 172.21.15.3:0/535151914 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1400343f0 con 0x7fb1480adbd0 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb14d0d4640 1 -- 172.21.15.3:0/535151914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480adbd0 msgr2=0x7fb148150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb14d0d4640 1 --2- 172.21.15.3:0/535151914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480adbd0 0x7fb148150a10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb1480bca80 tx=0x7fb14002be50 comp rx=0 tx=0).stop 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb14d0d4640 1 -- 172.21.15.3:0/535151914 shutdown_connections 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb14d0d4640 1 --2- 172.21.15.3:0/535151914 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480adbd0 0x7fb148150a10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb14d0d4640 1 --2- 172.21.15.3:0/535151914 >> v2:172.21.15.203:3300/0 conn(0x7fb1480ac300 0x7fb148149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb14d0d4640 1 -- 172.21.15.3:0/535151914 >> 172.21.15.3:0/535151914 conn(0x7fb1480aaa90 msgr2=0x7fb1480aaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb14d0d4640 1 -- 172.21.15.3:0/535151914 shutdown_connections 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.811+0000 7fb14d0d4640 1 -- 172.21.15.3:0/535151914 wait complete. 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14d0d4640 1 Processor -- start 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14d0d4640 1 -- start start 2024-05-21T17:41:25.817 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14d0d4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb14806cc60 0x7fb1480bc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14d0d4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480ac300 0x7fb14813d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14d0d4640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1480adbd0 0x7fb14813e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14d0d4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb1480ad450 con 0x7fb1480ac300 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14d0d4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb14814a4e0 con 0x7fb14806cc60 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14d0d4640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fb1480ad5d0 con 0x7fb1480adbd0 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480ac300 0x7fb14813d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14759e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1480adbd0 0x7fb14813e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480ac300 0x7fb14813d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50184/0 (socket says 172.21.15.3:50184) 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.813+0000 7fb14659c640 1 -- 172.21.15.3:0/288713612 learned_addr learned my addr 172.21.15.3:0/288713612 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb146d9d640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb14806cc60 0x7fb1480bc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb14659c640 1 -- 172.21.15.3:0/288713612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1480adbd0 msgr2=0x7fb14813e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb14659c640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1480adbd0 0x7fb14813e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.818 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb14659c640 1 -- 172.21.15.3:0/288713612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb14806cc60 msgr2=0x7fb1480bc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb14659c640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb14806cc60 0x7fb1480bc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb14659c640 1 -- 172.21.15.3:0/288713612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14813e140 con 0x7fb1480ac300 2024-05-21T17:41:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb14759e640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1480adbd0 0x7fb14813e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb14659c640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480ac300 0x7fb14813d860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00c3c0 tx=0x7fb13c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb12f7fe640 1 -- 172.21.15.3:0/288713612 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c017070 con 0x7fb1480ac300 2024-05-21T17:41:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb12f7fe640 1 -- 172.21.15.3:0/288713612 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb13c0052d0 con 0x7fb1480ac300 2024-05-21T17:41:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb14d0d4640 1 -- 172.21.15.3:0/288713612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14813f020 con 0x7fb1480ac300 2024-05-21T17:41:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb12f7fe640 1 -- 172.21.15.3:0/288713612 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c003830 con 0x7fb1480ac300 2024-05-21T17:41:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.814+0000 7fb14d0d4640 1 -- 172.21.15.3:0/288713612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1481d3190 con 0x7fb1480ac300 2024-05-21T17:41:25.819 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.815+0000 7fb14d0d4640 1 -- 172.21.15.3:0/288713612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14813cb30 con 0x7fb1480ac300 2024-05-21T17:41:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.818+0000 7fb12f7fe640 1 -- 172.21.15.3:0/288713612 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 146935+0+0 (secure 0 0 0) 0x7fb13c01a070 con 0x7fb1480ac300 2024-05-21T17:41:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.818+0000 7fb12f7fe640 20 mgrc handle_mgr_map mgrmap(e 4) 2024-05-21T17:41:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.818+0000 7fb12f7fe640 4 mgrc handle_mgr_map Got map version 4 2024-05-21T17:41:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.818+0000 7fb12f7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.818+0000 7fb12f7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:25.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.818+0000 7fb12f7fe640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb1340af1d0 0x7fb1340b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.819+0000 7fb12f7fe640 1 -- 172.21.15.3:0/288713612 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 2743+0+0 (secure 0 0 0) 0x7fb13c0db990 con 0x7fb1480ac300 2024-05-21T17:41:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.819+0000 7fb146d9d640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb1340af1d0 0x7fb1340b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:25.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.819+0000 7fb146d9d640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb1340af1d0 0x7fb1340b1690 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb13800a990 tx=0x7fb138008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:25.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:25.822+0000 7fb12f7fe640 1 -- 172.21.15.3:0/288713612 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb14813cb30 con 0x7fb1480ac300 2024-05-21T17:41:26.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.011+0000 7fb14d0d4640 1 -- 172.21.15.3:0/288713612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7fb148069a90 con 0x7fb1480ac300 2024-05-21T17:41:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.138+0000 7fb12f7fe640 1 -- 172.21.15.3:0/288713612 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v10) ==== 109+0+0 (secure 0 0 0) 0x7fb13c021020 con 0x7fb1480ac300 2024-05-21T17:41:26.140 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'rbd' created 2024-05-21T17:41:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 10 mgrc shutdown 2024-05-21T17:41:26.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 -- 172.21.15.3:0/288713612 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb1340af1d0 msgr2=0x7fb1340b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb1340af1d0 0x7fb1340b1690 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb13800a990 tx=0x7fb138008040 comp rx=0 tx=0).stop 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 -- 172.21.15.3:0/288713612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480ac300 msgr2=0x7fb14813d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480ac300 0x7fb14813d860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00c3c0 tx=0x7fb13c00c880 comp rx=0 tx=0).stop 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 -- 172.21.15.3:0/288713612 shutdown_connections 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb1340af1d0 0x7fb1340b1690 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb1480adbd0 0x7fb14813e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1480ac300 0x7fb14813d860 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 --2- 172.21.15.3:0/288713612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb14806cc60 0x7fb1480bc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 -- 172.21.15.3:0/288713612 >> 172.21.15.3:0/288713612 conn(0x7fb1480aaa90 msgr2=0x7fb1480a89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 -- 172.21.15.3:0/288713612 shutdown_connections 2024-05-21T17:41:26.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.140+0000 7fb14d0d4640 1 -- 172.21.15.3:0/288713612 wait complete. 2024-05-21T17:41:26.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-21T17:41:26.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.257+0000 7f6cff5e8640 1 Processor -- start 2024-05-21T17:41:26.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.258+0000 7f6cff5e8640 1 -- start start 2024-05-21T17:41:26.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.258+0000 7f6cff5e8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf80ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:26.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.258+0000 7f6cff5e8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6cf8146650 0x7f6cf8150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.258+0000 7f6cff5e8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf8146b20 con 0x7f6cf8149720 2024-05-21T17:41:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.258+0000 7f6cff5e8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf80a8530 con 0x7f6cf8146650 2024-05-21T17:41:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.258+0000 7f6cff5e8640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf8146de0 con 0x7f6cf80ac330 2024-05-21T17:41:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.258+0000 7f6cfd384640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf80ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.258+0000 7f6cfd384640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf80ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39532/0 (socket says 172.21.15.3:39532) 2024-05-21T17:41:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.258+0000 7f6cfd384640 1 -- 172.21.15.3:0/3236879489 learned_addr learned my addr 172.21.15.3:0/3236879489 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.259+0000 7f6cfd384640 1 -- 172.21.15.3:0/3236879489 >> v2:172.21.15.203:3300/0 conn(0x7f6cf8146650 msgr2=0x7f6cf8150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.259+0000 7f6cfdb85640 1 --2- 172.21.15.3:0/3236879489 >> v2:172.21.15.203:3300/0 conn(0x7f6cf8146650 0x7f6cf8150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:26.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.259+0000 7f6cfd384640 1 --2- 172.21.15.3:0/3236879489 >> v2:172.21.15.203:3300/0 conn(0x7f6cf8146650 0x7f6cf8150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.259+0000 7f6cfd384640 1 -- 172.21.15.3:0/3236879489 >> v1:172.21.15.203:6789/0 conn(0x7f6cf8149720 legacy=0x7f6cf8149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.259+0000 7f6cfd384640 1 -- 172.21.15.3:0/3236879489 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf80ac710 con 0x7f6cf80ac330 2024-05-21T17:41:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.259+0000 7f6cfd384640 1 --2- 172.21.15.3:0/3236879489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf80ad480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009870 tx=0x7f6cf402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e678e0107daf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6ceffff640 1 -- 172.21.15.3:0/3236879489 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40317c0 con 0x7f6cf80ac330 2024-05-21T17:41:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6ceffff640 1 -- 172.21.15.3:0/3236879489 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf4037070 con 0x7f6cf80ac330 2024-05-21T17:41:26.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6ceffff640 1 -- 172.21.15.3:0/3236879489 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4031b80 con 0x7f6cf80ac330 2024-05-21T17:41:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6cff5e8640 1 -- 172.21.15.3:0/3236879489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 msgr2=0x7f6cf80ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6cff5e8640 1 --2- 172.21.15.3:0/3236879489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf80ad480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009870 tx=0x7f6cf402b860 comp rx=0 tx=0).stop 2024-05-21T17:41:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6cff5e8640 1 -- 172.21.15.3:0/3236879489 shutdown_connections 2024-05-21T17:41:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6cff5e8640 1 --2- 172.21.15.3:0/3236879489 >> v2:172.21.15.203:3300/0 conn(0x7f6cf8146650 0x7f6cf8150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6cff5e8640 1 --2- 172.21.15.3:0/3236879489 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf80ad480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6cff5e8640 1 -- 172.21.15.3:0/3236879489 >> 172.21.15.3:0/3236879489 conn(0x7f6cf80aab00 msgr2=0x7f6cf80aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6cff5e8640 1 -- 172.21.15.3:0/3236879489 shutdown_connections 2024-05-21T17:41:26.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.260+0000 7f6cff5e8640 1 -- 172.21.15.3:0/3236879489 wait complete. 2024-05-21T17:41:26.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.261+0000 7f6cff5e8640 1 Processor -- start 2024-05-21T17:41:26.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.261+0000 7f6cff5e8640 1 -- start start 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cff5e8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf81d2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cff5e8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf8146650 0x7f6cf8140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cff5e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8149720 0x7f6cf813d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cff5e8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf80adec0 con 0x7f6cf8146650 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cff5e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf806cf00 con 0x7f6cf8149720 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cff5e8640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf814a210 con 0x7f6cf80ac330 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfd384640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf81d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfd384640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf81d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39534/0 (socket says 172.21.15.3:39534) 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfd384640 1 -- 172.21.15.3:0/1843098638 learned_addr learned my addr 172.21.15.3:0/1843098638 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfcb83640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf8146650 0x7f6cf8140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:26.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfdb85640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8149720 0x7f6cf813d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfd384640 1 -- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8149720 msgr2=0x7f6cf813d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfd384640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8149720 0x7f6cf813d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfd384640 1 -- 172.21.15.3:0/1843098638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf8146650 msgr2=0x7f6cf8140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfd384640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf8146650 0x7f6cf8140ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfd384640 1 -- 172.21.15.3:0/1843098638 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf813df90 con 0x7f6cf80ac330 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.262+0000 7f6cfdb85640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8149720 0x7f6cf813d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.263+0000 7f6cfd384640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf81d2310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6cf402b8a0 tx=0x7f6cf40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.263+0000 7f6cedffb640 1 -- 172.21.15.3:0/1843098638 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4032860 con 0x7f6cf80ac330 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.263+0000 7f6cedffb640 1 -- 172.21.15.3:0/1843098638 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf4032a20 con 0x7f6cf80ac330 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.263+0000 7f6cedffb640 1 -- 172.21.15.3:0/1843098638 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4032e10 con 0x7f6cf80ac330 2024-05-21T17:41:26.270 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.263+0000 7f6cff5e8640 1 -- 172.21.15.3:0/1843098638 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf813e250 con 0x7f6cf80ac330 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.263+0000 7f6cff5e8640 1 -- 172.21.15.3:0/1843098638 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf81411e0 con 0x7f6cf80ac330 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.264+0000 7f6cff5e8640 1 -- 172.21.15.3:0/1843098638 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf8069a90 con 0x7f6cf80ac330 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.265+0000 7f6cedffb640 1 -- 172.21.15.3:0/1843098638 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 4) ==== 146935+0+0 (secure 0 0 0) 0x7f6cf40453f0 con 0x7f6cf80ac330 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.265+0000 7f6cedffb640 20 mgrc handle_mgr_map mgrmap(e 4) 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.266+0000 7f6cedffb640 4 mgrc handle_mgr_map Got map version 4 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.266+0000 7f6cedffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.266+0000 7f6cedffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.266+0000 7f6cedffb640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f6cdc0af1d0 0x7f6cdc0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.266+0000 7f6cedffb640 1 -- 172.21.15.3:0/1843098638 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 3460+0+0 (secure 0 0 0) 0x7f6cf40fe660 con 0x7f6cf80ac330 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.266+0000 7f6cfcb83640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f6cdc0af1d0 0x7f6cdc0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:26.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.267+0000 7f6cfcb83640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f6cdc0af1d0 0x7f6cdc0b1690 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6cf81475d0 tx=0x7f6ce8009060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:26.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.270+0000 7f6cedffb640 1 -- 172.21.15.3:0/1843098638 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6cf4081990 con 0x7f6cf80ac330 2024-05-21T17:41:26.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:26.460+0000 7f6cff5e8640 1 -- 172.21.15.3:0/1843098638 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7f6cf8148900 con 0x7f6cf80ac330 2024-05-21T17:41:26.738 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-21T17:41:26.741 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-21T17:41:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.146+0000 7f6cedffb640 1 -- 172.21.15.3:0/1843098638 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v11) ==== 173+0+0 (secure 0 0 0) 0x7f6cf4040070 con 0x7f6cf80ac330 2024-05-21T17:41:27.148 INFO:teuthology.orchestra.run.smithi003.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-21T17:41:27.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 10 mgrc shutdown 2024-05-21T17:41:27.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 -- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f6cdc0af1d0 msgr2=0x7f6cdc0b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:27.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f6cdc0af1d0 0x7f6cdc0b1690 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6cf81475d0 tx=0x7f6ce8009060 comp rx=0 tx=0).stop 2024-05-21T17:41:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 -- 172.21.15.3:0/1843098638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 msgr2=0x7f6cf81d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf81d2310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6cf402b8a0 tx=0x7f6cf40046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 -- 172.21.15.3:0/1843098638 shutdown_connections 2024-05-21T17:41:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f6cdc0af1d0 0x7f6cdc0b1690 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6cf8149720 0x7f6cf813d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6cf8146650 0x7f6cf8140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 --2- 172.21.15.3:0/1843098638 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f6cf80ac330 0x7f6cf81d2310 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 -- 172.21.15.3:0/1843098638 >> 172.21.15.3:0/1843098638 conn(0x7f6cf80aab00 msgr2=0x7f6cf80bc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.149+0000 7f6cff5e8640 1 -- 172.21.15.3:0/1843098638 shutdown_connections 2024-05-21T17:41:27.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.150+0000 7f6cff5e8640 1 -- 172.21.15.3:0/1843098638 wait complete. 2024-05-21T17:41:27.166 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-21T17:41:27.166 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-21T17:41:27.166 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-21T17:41:27.169 INFO:tasks.ceph.mds.a:Started 2024-05-21T17:41:27.169 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-21T17:41:27.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-21T17:41:27.171 INFO:tasks.ceph.mds.c:Started 2024-05-21T17:41:27.171 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-21T17:41:27.171 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-21T17:41:27.174 INFO:tasks.ceph.mds.b:Started 2024-05-21T17:41:27.174 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-21T17:41:27.174 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T17:41:27.253 INFO:tasks.ceph.mds.a.smithi003.stdout:starting mds.a at 2024-05-21T17:41:27.253 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-05-21T17:41:27.253 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-05-21T17:41:27.251+0000 7fcc50a27640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-05-21T17:41:27.259 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-05-21T17:41:27.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.286+0000 7f1a65abc640 1 Processor -- start 2024-05-21T17:41:27.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.286+0000 7f1a65abc640 1 -- start start 2024-05-21T17:41:27.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a65abc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a60145ba0 0x7f1a600ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:27.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a65abc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1a6006cb50 0x7f1a601509e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:27.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a65abc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a60146990 con 0x7f1a6006c770 2024-05-21T17:41:27.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a65abc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f1a60146c30 con 0x7f1a60145ba0 2024-05-21T17:41:27.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a65abc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f1a60146690 con 0x7f1a6006cb50 2024-05-21T17:41:27.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a5f7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f1a60145ba0 0x7f1a600ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a5effd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f1a6006c770 0x7f1a60145f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35740/0 (socket says 172.21.15.3:35740) 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a5effd640 1 -- 172.21.15.3:0/1274097113 learned_addr learned my addr 172.21.15.3:0/1274097113 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a5ffff640 1 --2- 172.21.15.3:0/1274097113 >> v2:172.21.15.203:3300/0 conn(0x7f1a6006cb50 0x7f1a601509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a5f7fe640 1 -- 172.21.15.3:0/1274097113 >> v2:172.21.15.203:3300/0 conn(0x7f1a6006cb50 msgr2=0x7f1a601509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a5f7fe640 1 --2- 172.21.15.3:0/1274097113 >> v2:172.21.15.203:3300/0 conn(0x7f1a6006cb50 0x7f1a601509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a5f7fe640 1 -- 172.21.15.3:0/1274097113 >> v1:172.21.15.3:6789/0 conn(0x7f1a6006c770 legacy=0x7f1a60145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a5e7fc640 1 -- 172.21.15.3:0/1274097113 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3870850489 0 0) 0x7f1a60146990 con 0x7f1a6006c770 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.287+0000 7f1a5f7fe640 1 -- 172.21.15.3:0/1274097113 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6006d870 con 0x7f1a60145ba0 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.288+0000 7f1a5f7fe640 1 --2- 172.21.15.3:0/1274097113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a60145ba0 0x7f1a600ad430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1a600ad350 tx=0x7f1a5002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c1d9b6e673458c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.288+0000 7f1a5e7fc640 1 -- 172.21.15.3:0/1274097113 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50034070 con 0x7f1a60145ba0 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.288+0000 7f1a5e7fc640 1 -- 172.21.15.3:0/1274097113 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a50030610 con 0x7f1a60145ba0 2024-05-21T17:41:27.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.288+0000 7f1a5e7fc640 1 -- 172.21.15.3:0/1274097113 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500309e0 con 0x7f1a60145ba0 2024-05-21T17:41:27.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.289+0000 7f1a65abc640 1 -- 172.21.15.3:0/1274097113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a60145ba0 msgr2=0x7f1a600ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:27.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.289+0000 7f1a65abc640 1 --2- 172.21.15.3:0/1274097113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a60145ba0 0x7f1a600ad430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1a600ad350 tx=0x7f1a5002b960 comp rx=0 tx=0).stop 2024-05-21T17:41:27.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.289+0000 7f1a65abc640 1 -- 172.21.15.3:0/1274097113 shutdown_connections 2024-05-21T17:41:27.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.289+0000 7f1a65abc640 1 --2- 172.21.15.3:0/1274097113 >> v2:172.21.15.203:3300/0 conn(0x7f1a6006cb50 0x7f1a601509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.289+0000 7f1a65abc640 1 --2- 172.21.15.3:0/1274097113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a60145ba0 0x7f1a600ad430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.289+0000 7f1a65abc640 1 -- 172.21.15.3:0/1274097113 >> 172.21.15.3:0/1274097113 conn(0x7f1a600bbc20 msgr2=0x7f1a600bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:27.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.289+0000 7f1a65abc640 1 -- 172.21.15.3:0/1274097113 shutdown_connections 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.289+0000 7f1a65abc640 1 -- 172.21.15.3:0/1274097113 wait complete. 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.290+0000 7f1a65abc640 1 Processor -- start 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.290+0000 7f1a65abc640 1 -- start start 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a65abc640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a6006c770 0x7f1a600b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a65abc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a60145ba0 0x7f1a60147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a65abc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a600a0cb0 0x7f1a60148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a65abc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1a600bc890 con 0x7f1a600a0cb0 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a65abc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1a600bc710 con 0x7f1a60145ba0 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a65abc640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f1a60150f20 con 0x7f1a6006c770 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a600a0cb0 0x7f1a60148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5f7fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a6006c770 0x7f1a600b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5f7fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a6006c770 0x7f1a600b9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39582/0 (socket says 172.21.15.3:39582) 2024-05-21T17:41:27.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a600a0cb0 0x7f1a60148450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50238/0 (socket says 172.21.15.3:50238) 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5f7fe640 1 -- 172.21.15.3:0/2286889738 learned_addr learned my addr 172.21.15.3:0/2286889738 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5effd640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a60145ba0 0x7f1a60147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5ffff640 1 -- 172.21.15.3:0/2286889738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a6006c770 msgr2=0x7f1a600b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5ffff640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a6006c770 0x7f1a600b9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5ffff640 1 -- 172.21.15.3:0/2286889738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a60145ba0 msgr2=0x7f1a60147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5ffff640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a60145ba0 0x7f1a60147f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5ffff640 1 -- 172.21.15.3:0/2286889738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a600ba7e0 con 0x7f1a600a0cb0 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5f7fe640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a6006c770 0x7f1a600b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a5ffff640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a600a0cb0 0x7f1a60148450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800ef90 tx=0x7f1a5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a47fff640 1 -- 172.21.15.3:0/2286889738 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58018020 con 0x7f1a600a0cb0 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a47fff640 1 -- 172.21.15.3:0/2286889738 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a58014480 con 0x7f1a600a0cb0 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a47fff640 1 -- 172.21.15.3:0/2286889738 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58014870 con 0x7f1a600a0cb0 2024-05-21T17:41:27.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a65abc640 1 -- 172.21.15.3:0/2286889738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a60147990 con 0x7f1a600a0cb0 2024-05-21T17:41:27.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a65abc640 1 -- 172.21.15.3:0/2286889738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6014a400 con 0x7f1a600a0cb0 2024-05-21T17:41:27.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.292+0000 7f1a65abc640 1 -- 172.21.15.3:0/2286889738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a60069a90 con 0x7f1a600a0cb0 2024-05-21T17:41:27.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.294+0000 7f1a47fff640 1 -- 172.21.15.3:0/2286889738 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 146935+0+0 (secure 0 0 0) 0x7f1a58005d10 con 0x7f1a600a0cb0 2024-05-21T17:41:27.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.294+0000 7f1a47fff640 20 mgrc handle_mgr_map mgrmap(e 4) 2024-05-21T17:41:27.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.295+0000 7f1a47fff640 4 mgrc handle_mgr_map Got map version 4 2024-05-21T17:41:27.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.295+0000 7f1a47fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:27.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.295+0000 7f1a47fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:27.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.295+0000 7f1a47fff640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f1a4c0af1d0 0x7f1a4c0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:27.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.295+0000 7f1a5f7fe640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f1a4c0af1d0 0x7f1a4c0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:27.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.295+0000 7f1a47fff640 1 -- 172.21.15.3:0/2286889738 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 3482+0+0 (secure 0 0 0) 0x7f1a580dd800 con 0x7f1a600a0cb0 2024-05-21T17:41:27.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.295+0000 7f1a5f7fe640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f1a4c0af1d0 0x7f1a4c0b1690 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009560 tx=0x7f1a50008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:27.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.298+0000 7f1a47fff640 1 -- 172.21.15.3:0/2286889738 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1a58022030 con 0x7f1a600a0cb0 2024-05-21T17:41:27.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.534+0000 7f1a65abc640 1 -- 172.21.15.3:0/2286889738 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a6006d5c0 con 0x7f1a600a0cb0 2024-05-21T17:41:27.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.535+0000 7f1a47fff640 1 -- 172.21.15.3:0/2286889738 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1386 (secure 0 0 0) 0x7f1a58010040 con 0x7f1a600a0cb0 2024-05-21T17:41:27.537 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:27.537 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6810/205822275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":205822275},{"type":"v1","addr":"172.21.15.3:6810","nonce":205822275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-21T17:41:27.537 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 2 2024-05-21T17:41:27.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 10 mgrc shutdown 2024-05-21T17:41:27.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 -- 172.21.15.3:0/2286889738 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f1a4c0af1d0 msgr2=0x7f1a4c0b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:27.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f1a4c0af1d0 0x7f1a4c0b1690 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009560 tx=0x7f1a50008030 comp rx=0 tx=0).stop 2024-05-21T17:41:27.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 -- 172.21.15.3:0/2286889738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a600a0cb0 msgr2=0x7f1a60148450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a600a0cb0 0x7f1a60148450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800ef90 tx=0x7f1a5800c490 comp rx=0 tx=0).stop 2024-05-21T17:41:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 -- 172.21.15.3:0/2286889738 shutdown_connections 2024-05-21T17:41:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f1a4c0af1d0 0x7f1a4c0b1690 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1a600a0cb0 0x7f1a60148450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a60145ba0 0x7f1a60147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 --2- 172.21.15.3:0/2286889738 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f1a6006c770 0x7f1a600b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 -- 172.21.15.3:0/2286889738 >> 172.21.15.3:0/2286889738 conn(0x7f1a600bbc20 msgr2=0x7f1a600a9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 -- 172.21.15.3:0/2286889738 shutdown_connections 2024-05-21T17:41:27.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:27.537+0000 7f1a65abc640 1 -- 172.21.15.3:0/2286889738 wait complete. 2024-05-21T17:41:29.552 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T17:41:29.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.666+0000 7f7d134ed640 1 Processor -- start 2024-05-21T17:41:29.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.666+0000 7f7d134ed640 1 -- start start 2024-05-21T17:41:29.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.667+0000 7f7d134ed640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7d0c0a6d10 0x7f7d0c14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:29.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.667+0000 7f7d134ed640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0c0a7990 con 0x7f7d0c0a7180 2024-05-21T17:41:29.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.667+0000 7f7d134ed640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0c074550 con 0x7f7d0c144cb0 2024-05-21T17:41:29.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.667+0000 7f7d134ed640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0c1440d0 con 0x7f7d0c0a6d10 2024-05-21T17:41:29.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.667+0000 7f7d11a8a640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f7d0c0a7180 0x7f7d0c14ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35748/0 (socket says 172.21.15.3:35748) 2024-05-21T17:41:29.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.667+0000 7f7d11a8a640 1 -- 172.21.15.3:0/1814617269 learned_addr learned my addr 172.21.15.3:0/1814617269 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:29.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.667+0000 7f7d10a88640 1 --2- 172.21.15.3:0/1814617269 >> v2:172.21.15.203:3300/0 conn(0x7f7d0c0a6d10 0x7f7d0c14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2021541866 0 0) 0x7f7d0c0a7990 con 0x7f7d0c0a7180 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cf8003520 con 0x7f7d0c0a7180 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 236467360 0 0) 0x7f7cf8003520 con 0x7f7d0c0a7180 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 >> v2:172.21.15.203:3300/0 conn(0x7f7d0c0a6d10 msgr2=0x7f7d0c14c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 --2- 172.21.15.3:0/1814617269 >> v2:172.21.15.203:3300/0 conn(0x7f7d0c0a6d10 0x7f7d0c14c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 >> v1:172.21.15.203:6789/0 conn(0x7f7d0c144cb0 legacy=0x7f7d0c073e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c0a6b00 con 0x7f7d0c0a7180 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3544809896 0 0) 0x7f7d040016f0 con 0x7f7d0c0a7180 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 >> v1:172.21.15.3:6789/0 conn(0x7f7d0c0a7180 legacy=0x7f7d0c14ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8003a50 0x7f7cf8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf8004800 0x7f7cf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:29.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cf80052e0 0x7f7cf800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1440d0 con 0x7f7cf8004800 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf800c1b0 con 0x7f7cf8003a50 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf800c3b0 con 0x7f7cf80052e0 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d10a88640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf8004800 0x7f7cf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d11289640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8003a50 0x7f7cf8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d11a8a640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cf80052e0 0x7f7cf800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d10a88640 1 -- 172.21.15.3:0/1814617269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cf80052e0 msgr2=0x7f7cf800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d10a88640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7cf80052e0 0x7f7cf800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d10a88640 1 -- 172.21.15.3:0/1814617269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8003a50 msgr2=0x7f7cf8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d10a88640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8003a50 0x7f7cf8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d10a88640 1 -- 172.21.15.3:0/1814617269 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf8003520 con 0x7f7cf8004800 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d11a8a640 1 -- 172.21.15.3:0/1814617269 reap_dead start 2024-05-21T17:41:29.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.668+0000 7f7d11289640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8003a50 0x7f7cf8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.669+0000 7f7d10a88640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf8004800 0x7f7cf8004c80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009a90 tx=0x7f7cf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4667027410096058 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.669+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc039070 con 0x7f7cf8004800 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.669+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc02c680 con 0x7f7cf8004800 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.669+0000 7f7d03fff640 1 -- 172.21.15.3:0/1814617269 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc009690 con 0x7f7cf8004800 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.670+0000 7f7d134ed640 1 -- 172.21.15.3:0/1814617269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf8004800 msgr2=0x7f7cf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.670+0000 7f7d134ed640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf8004800 0x7f7cf8004c80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009a90 tx=0x7f7cf8005830 comp rx=0 tx=0).stop 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.670+0000 7f7d134ed640 1 -- 172.21.15.3:0/1814617269 shutdown_connections 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.670+0000 7f7d134ed640 1 --2- 172.21.15.3:0/1814617269 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7cf8004800 0x7f7cf8004c80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.670+0000 7f7d134ed640 1 -- 172.21.15.3:0/1814617269 >> 172.21.15.3:0/1814617269 conn(0x7f7d0c0a8c40 msgr2=0x7f7d0c0a9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.670+0000 7f7d134ed640 1 -- 172.21.15.3:0/1814617269 shutdown_connections 2024-05-21T17:41:29.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.670+0000 7f7d134ed640 1 -- 172.21.15.3:0/1814617269 wait complete. 2024-05-21T17:41:29.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.671+0000 7f7d134ed640 1 Processor -- start 2024-05-21T17:41:29.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.671+0000 7f7d134ed640 1 -- start start 2024-05-21T17:41:29.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d134ed640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8004800 0x7f7d0c1cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:29.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d134ed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c1d9230 0x7f7d0c147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:29.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d134ed640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c1d9650 0x7f7d0c147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:29.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d134ed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c1445f0 con 0x7f7d0c1d9230 2024-05-21T17:41:29.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d134ed640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c144330 con 0x7f7cf8004800 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d134ed640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c0739c0 con 0x7f7d0c1d9650 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d10a88640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c1d9230 0x7f7d0c147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d11a8a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c1d9650 0x7f7d0c147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d11a8a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c1d9650 0x7f7d0c147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39596/0 (socket says 172.21.15.3:39596) 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d10a88640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c1d9230 0x7f7d0c147700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50250/0 (socket says 172.21.15.3:50250) 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d11a8a640 1 -- 172.21.15.3:0/1438350938 learned_addr learned my addr 172.21.15.3:0/1438350938 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d11289640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8004800 0x7f7d0c1cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d11a8a640 1 -- 172.21.15.3:0/1438350938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8004800 msgr2=0x7f7d0c1cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d11a8a640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8004800 0x7f7d0c1cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d11a8a640 1 -- 172.21.15.3:0/1438350938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c1d9230 msgr2=0x7f7d0c147700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d11a8a640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c1d9230 0x7f7d0c147700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:29.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d11a8a640 1 -- 172.21.15.3:0/1438350938 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c148180 con 0x7f7d0c1d9650 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.672+0000 7f7d10a88640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c1d9230 0x7f7d0c147700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.673+0000 7f7d11a8a640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c1d9650 0x7f7d0c147c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d04001050 tx=0x7f7d04007430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.673+0000 7f7d01ffb640 1 -- 172.21.15.3:0/1438350938 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0401a070 con 0x7f7d0c1d9650 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.673+0000 7f7d01ffb640 1 -- 172.21.15.3:0/1438350938 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0400f730 con 0x7f7d0c1d9650 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.673+0000 7f7d01ffb640 1 -- 172.21.15.3:0/1438350938 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400fb20 con 0x7f7d0c1d9650 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.673+0000 7f7d134ed640 1 -- 172.21.15.3:0/1438350938 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0c1d9c20 con 0x7f7d0c1d9650 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.673+0000 7f7d134ed640 1 -- 172.21.15.3:0/1438350938 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0c0a3210 con 0x7f7d0c1d9650 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.674+0000 7f7d134ed640 1 -- 172.21.15.3:0/1438350938 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0c0a3cb0 con 0x7f7d0c1d9650 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.675+0000 7f7d01ffb640 1 -- 172.21.15.3:0/1438350938 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f7d0401d070 con 0x7f7d0c1d9650 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.675+0000 7f7d01ffb640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.676+0000 7f7d01ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.676+0000 7f7d01ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.676+0000 7f7d01ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:29.679 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.676+0000 7f7d01ffb640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f7cf00af290 0x7f7cf00b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:29.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.676+0000 7f7d01ffb640 1 -- 172.21.15.3:0/1438350938 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f7d040dbc70 con 0x7f7d0c1d9650 2024-05-21T17:41:29.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.676+0000 7f7d11289640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f7cf00af290 0x7f7cf00b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:29.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.677+0000 7f7d11289640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f7cf00af290 0x7f7cf00b1750 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c066230 tx=0x7f7d08009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:29.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.681+0000 7f7d01ffb640 1 -- 172.21.15.3:0/1438350938 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7d040a5cd0 con 0x7f7d0c1d9650 2024-05-21T17:41:29.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.913+0000 7f7d134ed640 1 -- 172.21.15.3:0/1438350938 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d0c073c20 con 0x7f7d0c1d9650 2024-05-21T17:41:29.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.913+0000 7f7d01ffb640 1 -- 172.21.15.3:0/1438350938 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3034 (secure 0 0 0) 0x7f7d040a55d0 con 0x7f7d0c1d9650 2024-05-21T17:41:29.914 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:29.914 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/3741447020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3741447020},{"type":"v1","addr":"172.21.15.203:6827","nonce":3741447020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6810/205822275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":205822275},{"type":"v1","addr":"172.21.15.3:6810","nonce":205822275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4245,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6811/1903526925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6809","nonce":1903526925},{"type":"v1","addr":"172.21.15.3:6811","nonce":1903526925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-21T17:41:29.915 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2024-05-21T17:41:29.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 10 mgrc shutdown 2024-05-21T17:41:29.916 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 -- 172.21.15.3:0/1438350938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f7cf00af290 msgr2=0x7f7cf00b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f7cf00af290 0x7f7cf00b1750 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c066230 tx=0x7f7d08009040 comp rx=0 tx=0).stop 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 -- 172.21.15.3:0/1438350938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c1d9650 msgr2=0x7f7d0c147c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c1d9650 0x7f7d0c147c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d04001050 tx=0x7f7d04007430 comp rx=0 tx=0).stop 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 -- 172.21.15.3:0/1438350938 shutdown_connections 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f7cf00af290 0x7f7cf00b1750 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f7d0c1d9650 0x7f7d0c147c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d0c1d9230 0x7f7d0c147700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 --2- 172.21.15.3:0/1438350938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8004800 0x7f7d0c1cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 -- 172.21.15.3:0/1438350938 >> 172.21.15.3:0/1438350938 conn(0x7f7d0c0a8c40 msgr2=0x7f7d0c0aac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 -- 172.21.15.3:0/1438350938 shutdown_connections 2024-05-21T17:41:29.917 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:29.915+0000 7f7d134ed640 1 -- 172.21.15.3:0/1438350938 wait complete. 2024-05-21T17:41:29.927 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-21T17:41:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.029+0000 7f72e1325640 1 Processor -- start 2024-05-21T17:41:30.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.029+0000 7f72e1325640 1 -- start start 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72e1325640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72dc0ab060 0x7f72dc146a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72e1325640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72dc06c730 con 0x7f72dc072410 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72e1325640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72dc06c9f0 con 0x7f72dc146fa0 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72e1325640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f72dc074640 con 0x7f72dc0ab060 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72da7fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f72dc0ab060 0x7f72dc146a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72daffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f72dc072410 0x7f72dc0aa300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35752/0 (socket says 172.21.15.3:35752) 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72daffd640 1 -- 172.21.15.3:0/2526541301 learned_addr learned my addr 172.21.15.3:0/2526541301 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72da7fc640 1 -- 172.21.15.3:0/2526541301 >> v1:172.21.15.203:6789/0 conn(0x7f72dc146fa0 legacy=0x7f72dc150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72da7fc640 1 -- 172.21.15.3:0/2526541301 >> v1:172.21.15.3:6789/0 conn(0x7f72dc072410 legacy=0x7f72dc0aa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72da7fc640 1 -- 172.21.15.3:0/2526541301 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc0743c0 con 0x7f72dc0ab060 2024-05-21T17:41:30.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.030+0000 7f72d9ffb640 1 -- 172.21.15.3:0/2526541301 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4210726152 0 0) 0x7f72dc06c730 con 0x7f72dc072410 2024-05-21T17:41:30.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.031+0000 7f72da7fc640 1 --2- 172.21.15.3:0/2526541301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc0ab060 0x7f72dc146a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f72d00097a0 tx=0x7f72d002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429b8f31db3f03ba server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:30.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.031+0000 7f72d9ffb640 1 -- 172.21.15.3:0/2526541301 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0038070 con 0x7f72dc0ab060 2024-05-21T17:41:30.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.031+0000 7f72d9ffb640 1 -- 172.21.15.3:0/2526541301 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d0031700 con 0x7f72dc0ab060 2024-05-21T17:41:30.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.031+0000 7f72d9ffb640 1 -- 172.21.15.3:0/2526541301 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0031ad0 con 0x7f72dc0ab060 2024-05-21T17:41:30.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.032+0000 7f72e1325640 1 -- 172.21.15.3:0/2526541301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc0ab060 msgr2=0x7f72dc146a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.032+0000 7f72e1325640 1 --2- 172.21.15.3:0/2526541301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc0ab060 0x7f72dc146a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f72d00097a0 tx=0x7f72d002b7a0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.032+0000 7f72e1325640 1 -- 172.21.15.3:0/2526541301 shutdown_connections 2024-05-21T17:41:30.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.032+0000 7f72e1325640 1 --2- 172.21.15.3:0/2526541301 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc0ab060 0x7f72dc146a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.032+0000 7f72e1325640 1 -- 172.21.15.3:0/2526541301 >> 172.21.15.3:0/2526541301 conn(0x7f72dc071450 msgr2=0x7f72dc06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:30.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.032+0000 7f72e1325640 1 -- 172.21.15.3:0/2526541301 shutdown_connections 2024-05-21T17:41:30.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.032+0000 7f72e1325640 1 -- 172.21.15.3:0/2526541301 wait complete. 2024-05-21T17:41:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.033+0000 7f72e1325640 1 Processor -- start 2024-05-21T17:41:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.033+0000 7f72e1325640 1 -- start start 2024-05-21T17:41:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72e1325640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc072410 0x7f72dc145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72e1325640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0ab060 0x7f72dc13c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72e1325640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f72dc146fa0 0x7f72dc14b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72e1325640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc0b6490 con 0x7f72dc072410 2024-05-21T17:41:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72e1325640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc0727f0 con 0x7f72dc0ab060 2024-05-21T17:41:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72e1325640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f72dc0b6610 con 0x7f72dc146fa0 2024-05-21T17:41:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72daffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc072410 0x7f72dc145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:30.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72db7fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f72dc146fa0 0x7f72dc14b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72daffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc072410 0x7f72dc145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50272/0 (socket says 172.21.15.3:50272) 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72daffd640 1 -- 172.21.15.3:0/3252036419 learned_addr learned my addr 172.21.15.3:0/3252036419 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72da7fc640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0ab060 0x7f72dc13c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72daffd640 1 -- 172.21.15.3:0/3252036419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f72dc146fa0 msgr2=0x7f72dc14b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72daffd640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f72dc146fa0 0x7f72dc14b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72daffd640 1 -- 172.21.15.3:0/3252036419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0ab060 msgr2=0x7f72dc13c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72daffd640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0ab060 0x7f72dc13c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72daffd640 1 -- 172.21.15.3:0/3252036419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc14b7c0 con 0x7f72dc072410 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72db7fe640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f72dc146fa0 0x7f72dc14b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.034+0000 7f72daffd640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc072410 0x7f72dc145270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c410 tx=0x7f72cc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.035+0000 7f72c37fe640 1 -- 172.21.15.3:0/3252036419 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0074e0 con 0x7f72dc072410 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.035+0000 7f72c37fe640 1 -- 172.21.15.3:0/3252036419 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72cc0047e0 con 0x7f72dc072410 2024-05-21T17:41:30.038 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.035+0000 7f72e1325640 1 -- 172.21.15.3:0/3252036419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72dc1df6e0 con 0x7f72dc072410 2024-05-21T17:41:30.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.035+0000 7f72c37fe640 1 -- 172.21.15.3:0/3252036419 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc004bd0 con 0x7f72dc072410 2024-05-21T17:41:30.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.035+0000 7f72e1325640 1 -- 172.21.15.3:0/3252036419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72dc13bf90 con 0x7f72dc072410 2024-05-21T17:41:30.039 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.036+0000 7f72e1325640 1 -- 172.21.15.3:0/3252036419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72dc069290 con 0x7f72dc072410 2024-05-21T17:41:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.037+0000 7f72c37fe640 1 -- 172.21.15.3:0/3252036419 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f72cc011070 con 0x7f72dc072410 2024-05-21T17:41:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.037+0000 7f72c37fe640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.038+0000 7f72c37fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.038+0000 7f72c37fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.038+0000 7f72c37fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.038+0000 7f72c37fe640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f72c80af260 0x7f72c80b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.038+0000 7f72c37fe640 1 -- 172.21.15.3:0/3252036419 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f72cc0dc090 con 0x7f72dc072410 2024-05-21T17:41:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.038+0000 7f72da7fc640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f72c80af260 0x7f72c80b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:30.040 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.038+0000 7f72da7fc640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f72c80af260 0x7f72c80b1720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f72d0035000 tx=0x7f72d0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:30.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.042+0000 7f72c37fe640 1 -- 172.21.15.3:0/3252036419 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f72cc0a6190 con 0x7f72dc072410 2024-05-21T17:41:30.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.234+0000 7f72e1325640 1 -- 172.21.15.3:0/3252036419 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f72dc147da0 con 0x7f72dc072410 2024-05-21T17:41:30.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.234+0000 7f72c37fe640 1 -- 172.21.15.3:0/3252036419 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v12) ==== 84+0+126 (secure 0 0 0) 0x7f72cc08d070 con 0x7f72dc072410 2024-05-21T17:41:30.235 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:30.235 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-05-21T17:41:30.236 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-21T17:41:30.236 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-05-21T17:41:30.236 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-05-21T17:41:30.236 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-21T17:41:30.236 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-21T17:41:30.236 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 2, 2024-05-21T17:41:30.236 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": "rbd" 2024-05-21T17:41:30.236 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-21T17:41:30.236 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-05-21T17:41:30.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 10 mgrc shutdown 2024-05-21T17:41:30.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 -- 172.21.15.3:0/3252036419 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f72c80af260 msgr2=0x7f72c80b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f72c80af260 0x7f72c80b1720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f72d0035000 tx=0x7f72d0008030 comp rx=0 tx=0).stop 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 -- 172.21.15.3:0/3252036419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc072410 msgr2=0x7f72dc145270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc072410 0x7f72dc145270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c410 tx=0x7f72cc00c8d0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 -- 172.21.15.3:0/3252036419 shutdown_connections 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f72c80af260 0x7f72c80b1720 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f72dc146fa0 0x7f72dc14b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72dc0ab060 0x7f72dc13c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 --2- 172.21.15.3:0/3252036419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f72dc072410 0x7f72dc145270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 -- 172.21.15.3:0/3252036419 >> 172.21.15.3:0/3252036419 conn(0x7f72dc071450 msgr2=0x7f72dc0b7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 -- 172.21.15.3:0/3252036419 shutdown_connections 2024-05-21T17:41:30.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.236+0000 7f72e1325640 1 -- 172.21.15.3:0/3252036419 wait complete. 2024-05-21T17:41:30.248 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-21T17:41:30.248 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-21T17:41:30.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.355+0000 7f9fb7c5e640 1 Processor -- start 2024-05-21T17:41:30.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.355+0000 7f9fb7c5e640 1 -- start start 2024-05-21T17:41:30.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.355+0000 7f9fb7c5e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9fb0145ba0 0x7f9fb00ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:30.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.355+0000 7f9fb7c5e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fb0146990 con 0x7f9fb006cb50 2024-05-21T17:41:30.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.355+0000 7f9fb7c5e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fb0146c30 con 0x7f9fb006c770 2024-05-21T17:41:30.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.355+0000 7f9fb7c5e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f9fb0146690 con 0x7f9fb0145ba0 2024-05-21T17:41:30.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.355+0000 7f9fb61fb640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f9fb006cb50 0x7f9fb01509e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35766/0 (socket says 172.21.15.3:35766) 2024-05-21T17:41:30.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.355+0000 7f9fb61fb640 1 -- 172.21.15.3:0/4209188533 learned_addr learned my addr 172.21.15.3:0/4209188533 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:30.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.356+0000 7f9fb59fa640 1 --2- 172.21.15.3:0/4209188533 >> v2:172.21.15.203:3300/0 conn(0x7f9fb0145ba0 0x7f9fb00ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:30.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.356+0000 7f9fb49f8640 1 -- 172.21.15.3:0/4209188533 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1283161195 0 0) 0x7f9fb0146990 con 0x7f9fb006cb50 2024-05-21T17:41:30.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.356+0000 7f9fb49f8640 1 -- 172.21.15.3:0/4209188533 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f9c003520 con 0x7f9fb006cb50 2024-05-21T17:41:30.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.356+0000 7f9fb49f8640 1 -- 172.21.15.3:0/4209188533 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3791146871 0 0) 0x7f9f9c003520 con 0x7f9fb006cb50 2024-05-21T17:41:30.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.356+0000 7f9fb59fa640 1 -- 172.21.15.3:0/4209188533 >> v1:172.21.15.203:6789/0 conn(0x7f9fb006c770 legacy=0x7f9fb0145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.356+0000 7f9fb59fa640 1 -- 172.21.15.3:0/4209188533 >> v1:172.21.15.3:6789/0 conn(0x7f9fb006cb50 legacy=0x7f9fb01509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.356+0000 7f9fb59fa640 1 -- 172.21.15.3:0/4209188533 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb006d870 con 0x7f9fb0145ba0 2024-05-21T17:41:30.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.357+0000 7f9fb59fa640 1 --2- 172.21.15.3:0/4209188533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fb0145ba0 0x7f9fb00ad430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0097a0 tx=0x7f9fac02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1f41c843b2b8410 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:30.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.357+0000 7f9fb49f8640 1 -- 172.21.15.3:0/4209188533 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8001a90 con 0x7f9fb0145ba0 2024-05-21T17:41:30.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.357+0000 7f9fb49f8640 1 -- 172.21.15.3:0/4209188533 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fac037070 con 0x7f9fb0145ba0 2024-05-21T17:41:30.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.357+0000 7f9fb49f8640 1 -- 172.21.15.3:0/4209188533 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac031790 con 0x7f9fb0145ba0 2024-05-21T17:41:30.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.358+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/4209188533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fb0145ba0 msgr2=0x7f9fb00ad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.358+0000 7f9fb7c5e640 1 --2- 172.21.15.3:0/4209188533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fb0145ba0 0x7f9fb00ad430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0097a0 tx=0x7f9fac02b8b0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.358+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/4209188533 shutdown_connections 2024-05-21T17:41:30.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.358+0000 7f9fb7c5e640 1 --2- 172.21.15.3:0/4209188533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fb0145ba0 0x7f9fb00ad430 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.358+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/4209188533 >> 172.21.15.3:0/4209188533 conn(0x7f9fb00bbc20 msgr2=0x7f9fb00bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.358+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/4209188533 shutdown_connections 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.358+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/4209188533 wait complete. 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb7c5e640 1 Processor -- start 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb7c5e640 1 -- start start 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb7c5e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9fb006c770 0x7f9fb00a7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb7c5e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fb0145ba0 0x7f9fb01413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb7c5e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb00a46b0 0x7f9fb0140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb7c5e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb00bc890 con 0x7f9fb00a46b0 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb7c5e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb00bc710 con 0x7f9fb0145ba0 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb7c5e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb0152100 con 0x7f9fb006c770 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb61fb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb00a46b0 0x7f9fb0140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:30.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb59fa640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9fb006c770 0x7f9fb00a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb61fb640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb00a46b0 0x7f9fb0140a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50278/0 (socket says 172.21.15.3:50278) 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.359+0000 7f9fb61fb640 1 -- 172.21.15.3:0/1314821938 learned_addr learned my addr 172.21.15.3:0/1314821938 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fb51f9640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fb0145ba0 0x7f9fb01413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fb61fb640 1 -- 172.21.15.3:0/1314821938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9fb006c770 msgr2=0x7f9fb00a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fb61fb640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9fb006c770 0x7f9fb00a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fb61fb640 1 -- 172.21.15.3:0/1314821938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fb0145ba0 msgr2=0x7f9fb01413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fb61fb640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fb0145ba0 0x7f9fb01413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fb61fb640 1 -- 172.21.15.3:0/1314821938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb0141a80 con 0x7f9fb00a46b0 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fb61fb640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb00a46b0 0x7f9fb0140a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8002230 tx=0x7f9fa800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fa67fc640 1 -- 172.21.15.3:0/1314821938 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8018070 con 0x7f9fb00a46b0 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fa67fc640 1 -- 172.21.15.3:0/1314821938 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa800a5e0 con 0x7f9fb00a46b0 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/1314821938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb0141d40 con 0x7f9fb00a46b0 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fa67fc640 1 -- 172.21.15.3:0/1314821938 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa800a9d0 con 0x7f9fb00a46b0 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.360+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/1314821938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb00a4c00 con 0x7f9fb00a46b0 2024-05-21T17:41:30.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.361+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/1314821938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb01449f0 con 0x7f9fb00a46b0 2024-05-21T17:41:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.362+0000 7f9fa67fc640 1 -- 172.21.15.3:0/1314821938 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f9fa801b070 con 0x7f9fb00a46b0 2024-05-21T17:41:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.362+0000 7f9fa67fc640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:30.365 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.363+0000 7f9fa67fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:30.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.363+0000 7f9fa67fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:30.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.363+0000 7f9fa67fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:30.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.363+0000 7f9fa67fc640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f9f940af260 0x7f9f940b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:30.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.363+0000 7f9fa67fc640 1 -- 172.21.15.3:0/1314821938 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f9fa80dbfd0 con 0x7f9fb00a46b0 2024-05-21T17:41:30.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.364+0000 7f9fb59fa640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f9f940af260 0x7f9f940b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:30.366 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.364+0000 7f9fb59fa640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f9f940af260 0x7f9f940b1720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9fac036000 tx=0x7f9fac03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:30.367 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.366+0000 7f9fa67fc640 1 -- 172.21.15.3:0/1314821938 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9fb01449f0 con 0x7f9fb00a46b0 2024-05-21T17:41:30.557 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:30.556+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/1314821938 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9fb0140720 con 0x7f9fb00a46b0 2024-05-21T17:41:31.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.164+0000 7f9fa67fc640 1 -- 172.21.15.3:0/1314821938 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v13) ==== 138+0+0 (secure 0 0 0) 0x7f9fb0140720 con 0x7f9fb00a46b0 2024-05-21T17:41:31.165 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-05-21T17:41:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 10 mgrc shutdown 2024-05-21T17:41:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/1314821938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f9f940af260 msgr2=0x7f9f940b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f9f940af260 0x7f9f940b1720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9fac036000 tx=0x7f9fac03d040 comp rx=0 tx=0).stop 2024-05-21T17:41:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/1314821938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb00a46b0 msgr2=0x7f9fb0140a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb00a46b0 0x7f9fb0140a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8002230 tx=0x7f9fa800ccd0 comp rx=0 tx=0).stop 2024-05-21T17:41:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/1314821938 shutdown_connections 2024-05-21T17:41:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f9f940af260 0x7f9f940b1720 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9fb00a46b0 0x7f9fb0140a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:31.168 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fb0145ba0 0x7f9fb01413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:31.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 1 --2- 172.21.15.3:0/1314821938 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f9fb006c770 0x7f9fb00a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:31.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.166+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/1314821938 >> 172.21.15.3:0/1314821938 conn(0x7f9fb00bbc20 msgr2=0x7f9fb00a9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:31.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.167+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/1314821938 shutdown_connections 2024-05-21T17:41:31.169 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.167+0000 7f9fb7c5e640 1 -- 172.21.15.3:0/1314821938 wait complete. 2024-05-21T17:41:31.181 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-21T17:41:31.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.291+0000 7f8e540ca640 1 Processor -- start 2024-05-21T17:41:31.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.292+0000 7f8e540ca640 1 -- start start 2024-05-21T17:41:31.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.292+0000 7f8e540ca640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ac410 0x7f8e4c146320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:31.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.292+0000 7f8e540ca640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4c146c70 con 0x7f8e4c0adda0 2024-05-21T17:41:31.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.292+0000 7f8e540ca640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e4c146f30 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.292+0000 7f8e540ca640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4c0759a0 con 0x7f8e4c0ac410 2024-05-21T17:41:31.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.292+0000 7f8e51e66640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ac410 0x7f8e4c146320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.292+0000 7f8e51665640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f8e4c0adda0 0x7f8e4c0a7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35774/0 (socket says 172.21.15.3:35774) 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.292+0000 7f8e51665640 1 -- 172.21.15.3:0/1093714715 learned_addr learned my addr 172.21.15.3:0/1093714715 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.293+0000 7f8e51e66640 1 -- 172.21.15.3:0/1093714715 >> v1:172.21.15.203:6789/0 conn(0x7f8e4c0ad5a0 legacy=0x7f8e4c150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.293+0000 7f8e50e64640 1 -- 172.21.15.3:0/1093714715 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211308273 0 0) 0x7f8e4c146c70 con 0x7f8e4c0adda0 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.293+0000 7f8e51e66640 1 -- 172.21.15.3:0/1093714715 >> v1:172.21.15.3:6789/0 conn(0x7f8e4c0adda0 legacy=0x7f8e4c0a7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.293+0000 7f8e51e66640 1 -- 172.21.15.3:0/1093714715 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c0ac7f0 con 0x7f8e4c0ac410 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.293+0000 7f8e51e66640 1 --2- 172.21.15.3:0/1093714715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ac410 0x7f8e4c146320 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8e480097d0 tx=0x7f8e4802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abd4dd9098ebbab7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.294+0000 7f8e50e64640 1 -- 172.21.15.3:0/1093714715 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48031760 con 0x7f8e4c0ac410 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.294+0000 7f8e50e64640 1 -- 172.21.15.3:0/1093714715 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e48037070 con 0x7f8e4c0ac410 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.294+0000 7f8e50e64640 1 -- 172.21.15.3:0/1093714715 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48031b20 con 0x7f8e4c0ac410 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.294+0000 7f8e540ca640 1 -- 172.21.15.3:0/1093714715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ac410 msgr2=0x7f8e4c146320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.294+0000 7f8e540ca640 1 --2- 172.21.15.3:0/1093714715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ac410 0x7f8e4c146320 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8e480097d0 tx=0x7f8e4802b800 comp rx=0 tx=0).stop 2024-05-21T17:41:31.297 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.294+0000 7f8e540ca640 1 -- 172.21.15.3:0/1093714715 shutdown_connections 2024-05-21T17:41:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.294+0000 7f8e540ca640 1 --2- 172.21.15.3:0/1093714715 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ac410 0x7f8e4c146320 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.294+0000 7f8e540ca640 1 -- 172.21.15.3:0/1093714715 >> 172.21.15.3:0/1093714715 conn(0x7f8e4c0bb9e0 msgr2=0x7f8e4c0bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.294+0000 7f8e540ca640 1 -- 172.21.15.3:0/1093714715 shutdown_connections 2024-05-21T17:41:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.294+0000 7f8e540ca640 1 -- 172.21.15.3:0/1093714715 wait complete. 2024-05-21T17:41:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.295+0000 7f8e540ca640 1 Processor -- start 2024-05-21T17:41:31.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.295+0000 7f8e540ca640 1 -- start start 2024-05-21T17:41:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e540ca640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e4c0ac410 0x7f8e4c09ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e540ca640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ad5a0 0x7f8e4c09e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e540ca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4c0adda0 0x7f8e4c09e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e540ca640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1469e0 con 0x7f8e4c0adda0 2024-05-21T17:41:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e540ca640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c146860 con 0x7f8e4c0ac410 2024-05-21T17:41:31.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e540ca640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4c0a83f0 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e51665640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ad5a0 0x7f8e4c09e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e52667640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4c0adda0 0x7f8e4c09e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e52667640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4c0adda0 0x7f8e4c09e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50280/0 (socket says 172.21.15.3:50280) 2024-05-21T17:41:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e51665640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ad5a0 0x7f8e4c09e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39640/0 (socket says 172.21.15.3:39640) 2024-05-21T17:41:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e52667640 1 -- 172.21.15.3:0/2278417283 learned_addr learned my addr 172.21.15.3:0/2278417283 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e51e66640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e4c0ac410 0x7f8e4c09ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e51665640 1 -- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e4c0ac410 msgr2=0x7f8e4c09ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e51665640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e4c0ac410 0x7f8e4c09ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e51665640 1 -- 172.21.15.3:0/2278417283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4c0adda0 msgr2=0x7f8e4c09e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:31.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e51665640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4c0adda0 0x7f8e4c09e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e51665640 1 -- 172.21.15.3:0/2278417283 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c1dbb00 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e52667640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4c0adda0 0x7f8e4c09e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.296+0000 7f8e51e66640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e4c0ac410 0x7f8e4c09ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.297+0000 7f8e51665640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ad5a0 0x7f8e4c09e150 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c390 tx=0x7f8e3c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.297+0000 7f8e427fc640 1 -- 172.21.15.3:0/2278417283 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c0074e0 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.297+0000 7f8e427fc640 1 -- 172.21.15.3:0/2278417283 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e3c0047e0 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.297+0000 7f8e427fc640 1 -- 172.21.15.3:0/2278417283 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c004bd0 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.297+0000 7f8e540ca640 1 -- 172.21.15.3:0/2278417283 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c144910 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.297+0000 7f8e540ca640 1 -- 172.21.15.3:0/2278417283 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1450d0 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.298+0000 7f8e540ca640 1 -- 172.21.15.3:0/2278417283 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e4c069a90 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.299+0000 7f8e427fc640 1 -- 172.21.15.3:0/2278417283 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f8e3c011070 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.299+0000 7f8e427fc640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:31.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.300+0000 7f8e427fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:31.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.300+0000 7f8e427fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:31.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.300+0000 7f8e427fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:31.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.300+0000 7f8e427fc640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f8e300af260 0x7f8e300b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:31.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.300+0000 7f8e427fc640 1 -- 172.21.15.3:0/2278417283 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(13..13 src has 1..13) ==== 3852+0+0 (secure 0 0 0) 0x7f8e3c0dc850 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.300+0000 7f8e51e66640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f8e300af260 0x7f8e300b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:31.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.301+0000 7f8e51e66640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f8e300af260 0x7f8e300b1720 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8e48036000 tx=0x7f8e48008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:31.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.303+0000 7f8e427fc640 1 -- 172.21.15.3:0/2278417283 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8e4c1dbb00 con 0x7f8e4c0ad5a0 2024-05-21T17:41:31.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:31.491+0000 7f8e540ca640 1 -- 172.21.15.3:0/2278417283 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8e4c06f210 con 0x7f8e4c0ad5a0 2024-05-21T17:41:32.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.170+0000 7f8e427fc640 1 -- 172.21.15.3:0/2278417283 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v14) ==== 170+0+0 (secure 0 0 0) 0x7f8e3c0a67e0 con 0x7f8e4c0ad5a0 2024-05-21T17:41:32.171 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-05-21T17:41:32.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.174+0000 7f8e540ca640 10 mgrc shutdown 2024-05-21T17:41:32.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.174+0000 7f8e540ca640 1 -- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f8e300af260 msgr2=0x7f8e300b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.174+0000 7f8e540ca640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f8e300af260 0x7f8e300b1720 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8e48036000 tx=0x7f8e48008030 comp rx=0 tx=0).stop 2024-05-21T17:41:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.174+0000 7f8e540ca640 1 -- 172.21.15.3:0/2278417283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ad5a0 msgr2=0x7f8e4c09e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.174+0000 7f8e540ca640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ad5a0 0x7f8e4c09e150 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c390 tx=0x7f8e3c00c850 comp rx=0 tx=0).stop 2024-05-21T17:41:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.175+0000 7f8e540ca640 1 -- 172.21.15.3:0/2278417283 shutdown_connections 2024-05-21T17:41:32.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.175+0000 7f8e540ca640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f8e300af260 0x7f8e300b1720 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8e48036000 tx=0x7f8e48008030 comp rx=0 tx=0).stop 2024-05-21T17:41:32.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.175+0000 7f8e540ca640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8e4c0adda0 0x7f8e4c09e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:32.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.175+0000 7f8e540ca640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f8e4c0ad5a0 0x7f8e4c09e150 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00c390 tx=0x7f8e3c00c850 comp rx=0 tx=0).stop 2024-05-21T17:41:32.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.175+0000 7f8e540ca640 1 --2- 172.21.15.3:0/2278417283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e4c0ac410 0x7f8e4c09ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:32.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.175+0000 7f8e540ca640 1 -- 172.21.15.3:0/2278417283 >> 172.21.15.3:0/2278417283 conn(0x7f8e4c0bb9e0 msgr2=0x7f8e4c0a7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:32.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.175+0000 7f8e540ca640 1 -- 172.21.15.3:0/2278417283 shutdown_connections 2024-05-21T17:41:32.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.176+0000 7f8e540ca640 1 -- 172.21.15.3:0/2278417283 wait complete. 2024-05-21T17:41:32.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-21T17:41:32.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.304+0000 7f8865aca640 1 Processor -- start 2024-05-21T17:41:32.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.304+0000 7f8865aca640 1 -- start start 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f8865aca640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f88600ad480 0x7f8860151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f8865aca640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88600ad8d0 con 0x7f88600ac330 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f8865aca640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f886014a540 con 0x7f8860146650 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f8865aca640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f8860146b20 con 0x7f88600ad480 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f88600ac330 0x7f8860149720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35776/0 (socket says 172.21.15.3:35776) 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885f7fe640 1 -- 172.21.15.3:0/1109106012 learned_addr learned my addr 172.21.15.3:0/1109106012 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885ffff640 1 --2- 172.21.15.3:0/1109106012 >> v2:172.21.15.203:3300/0 conn(0x7f88600ad480 0x7f8860151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2355840179 0 0) 0x7f88600ad8d0 con 0x7f88600ac330 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f884c003520 con 0x7f88600ac330 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 988662298 0 0) 0x7f884c003520 con 0x7f88600ac330 2024-05-21T17:41:32.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 >> v2:172.21.15.203:3300/0 conn(0x7f88600ad480 msgr2=0x7f8860151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 --2- 172.21.15.3:0/1109106012 >> v2:172.21.15.203:3300/0 conn(0x7f88600ad480 0x7f8860151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 >> v1:172.21.15.203:6789/0 conn(0x7f8860146650 legacy=0x7f8860149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8860146de0 con 0x7f88600ac330 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3544809896 0 0) 0x7f88500016f0 con 0x7f88600ac330 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 >> v1:172.21.15.3:6789/0 conn(0x7f88600ac330 legacy=0x7f8860149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f884c003a50 0x7f884c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c004800 0x7f884c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0052e0 0x7f884c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8860146b20 con 0x7f884c0052e0 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f884c00c1b0 con 0x7f884c003a50 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.305+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f884c00c3b0 con 0x7f884c004800 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.306+0000 7f885f7fe640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f884c003a50 0x7f884c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.306+0000 7f885ffff640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0052e0 0x7f884c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:32.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.306+0000 7f8857fff640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c004800 0x7f884c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.306+0000 7f885ffff640 1 -- 172.21.15.3:0/1109106012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c004800 msgr2=0x7f884c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.306+0000 7f885ffff640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f884c004800 0x7f884c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.306+0000 7f885ffff640 1 -- 172.21.15.3:0/1109106012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f884c003a50 msgr2=0x7f884c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.306+0000 7f885ffff640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f884c003a50 0x7f884c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.306+0000 7f885ffff640 1 -- 172.21.15.3:0/1109106012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c003520 con 0x7f884c0052e0 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.306+0000 7f885f7fe640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f884c003a50 0x7f884c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.307+0000 7f885ffff640 1 -- 172.21.15.3:0/1109106012 reap_dead start 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.307+0000 7f885ffff640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0052e0 0x7f884c00b9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8858009f90 tx=0x7f884c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b950f744006a15 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.307+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850007d70 con 0x7f884c0052e0 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.307+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f885802cc20 con 0x7f884c0052e0 2024-05-21T17:41:32.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.307+0000 7f885effd640 1 -- 172.21.15.3:0/1109106012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858009c10 con 0x7f884c0052e0 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.307+0000 7f8865aca640 1 -- 172.21.15.3:0/1109106012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0052e0 msgr2=0x7f884c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.307+0000 7f8865aca640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0052e0 0x7f884c00b9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8858009f90 tx=0x7f884c005830 comp rx=0 tx=0).stop 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.307+0000 7f8865aca640 1 -- 172.21.15.3:0/1109106012 shutdown_connections 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.307+0000 7f8865aca640 1 --2- 172.21.15.3:0/1109106012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f884c0052e0 0x7f884c00b9c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.307+0000 7f8865aca640 1 -- 172.21.15.3:0/1109106012 >> 172.21.15.3:0/1109106012 conn(0x7f88600aab00 msgr2=0x7f88600aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.308+0000 7f8865aca640 1 -- 172.21.15.3:0/1109106012 shutdown_connections 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.308+0000 7f8865aca640 1 -- 172.21.15.3:0/1109106012 wait complete. 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f8865aca640 1 Processor -- start 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f8865aca640 1 -- start start 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f8865aca640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f884c0052e0 0x7f886013ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:32.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f8865aca640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88601383d0 0x7f88601d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f8865aca640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88601d1f10 0x7f88601cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f8865aca640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f88600add80 con 0x7f88601d1f10 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f8865aca640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f886006ccf0 con 0x7f884c0052e0 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f8865aca640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f88601df6b0 con 0x7f88601383d0 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f885ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88601d1f10 0x7f88601cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f8857fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88601383d0 0x7f88601d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f885ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88601d1f10 0x7f88601cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50288/0 (socket says 172.21.15.3:50288) 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f885ffff640 1 -- 172.21.15.3:0/885120812 learned_addr learned my addr 172.21.15.3:0/885120812 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f8857fff640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88601383d0 0x7f88601d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39658/0 (socket says 172.21.15.3:39658) 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.309+0000 7f885f7fe640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f884c0052e0 0x7f886013ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f8857fff640 1 -- 172.21.15.3:0/885120812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f884c0052e0 msgr2=0x7f886013ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:32.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f8857fff640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f884c0052e0 0x7f886013ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f8857fff640 1 -- 172.21.15.3:0/885120812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88601d1f10 msgr2=0x7f88601cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f8857fff640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88601d1f10 0x7f88601cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f8857fff640 1 -- 172.21.15.3:0/885120812 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88601df970 con 0x7f88601383d0 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f885ffff640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88601d1f10 0x7f88601cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f8857fff640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88601383d0 0x7f88601d19d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f884800f650 tx=0x7f884800fb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f885cff9640 1 -- 172.21.15.3:0/885120812 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884801a070 con 0x7f88601383d0 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f885cff9640 1 -- 172.21.15.3:0/885120812 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8848005110 con 0x7f88601383d0 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f885cff9640 1 -- 172.21.15.3:0/885120812 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848007050 con 0x7f88601383d0 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f8865aca640 1 -- 172.21.15.3:0/885120812 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886013c3f0 con 0x7f88601383d0 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.310+0000 7f8865aca640 1 -- 172.21.15.3:0/885120812 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f88601372b0 con 0x7f88601383d0 2024-05-21T17:41:32.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.311+0000 7f8865aca640 1 -- 172.21.15.3:0/885120812 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f886006cac0 con 0x7f88601383d0 2024-05-21T17:41:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.312+0000 7f885cff9640 1 -- 172.21.15.3:0/885120812 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f88480071f0 con 0x7f88601383d0 2024-05-21T17:41:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.312+0000 7f885cff9640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.313+0000 7f885cff9640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.313+0000 7f885cff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.313+0000 7f885cff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.313+0000 7f885cff9640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f88400af290 0x7f88400b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.313+0000 7f885cff9640 1 -- 172.21.15.3:0/885120812 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 4234+0+0 (secure 0 0 0) 0x7f88480dbf00 con 0x7f88601383d0 2024-05-21T17:41:32.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.313+0000 7f885f7fe640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f88400af290 0x7f88400b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:32.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.314+0000 7f885f7fe640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f88400af290 0x7f88400b1750 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8850007cd0 tx=0x7f8850007970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:32.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.317+0000 7f885cff9640 1 -- 172.21.15.3:0/885120812 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f886006cac0 con 0x7f88601383d0 2024-05-21T17:41:32.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:32.551+0000 7f8865aca640 1 -- 172.21.15.3:0/885120812 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8860069a90 con 0x7f88601383d0 2024-05-21T17:41:33.182 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 3 and data pool 4 2024-05-21T17:41:33.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.180+0000 7f885cff9640 1 -- 172.21.15.3:0/885120812 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) ==== 172+0+0 (secure 0 0 0) 0x7f88480a5620 con 0x7f88601383d0 2024-05-21T17:41:33.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.184+0000 7f8865aca640 10 mgrc shutdown 2024-05-21T17:41:33.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.185+0000 7f8865aca640 1 -- 172.21.15.3:0/885120812 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f88400af290 msgr2=0x7f88400b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.185+0000 7f8865aca640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f88400af290 0x7f88400b1750 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8850007cd0 tx=0x7f8850007970 comp rx=0 tx=0).stop 2024-05-21T17:41:33.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.185+0000 7f8865aca640 1 -- 172.21.15.3:0/885120812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88601383d0 msgr2=0x7f88601d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.185+0000 7f8865aca640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88601383d0 0x7f88601d19d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f884800f650 tx=0x7f884800fb10 comp rx=0 tx=0).stop 2024-05-21T17:41:33.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.185+0000 7f8865aca640 1 -- 172.21.15.3:0/885120812 shutdown_connections 2024-05-21T17:41:33.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.185+0000 7f8865aca640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f88400af290 0x7f88400b1750 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.185+0000 7f8865aca640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f88601d1f10 0x7f88601cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.185+0000 7f8865aca640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f88601383d0 0x7f88601d19d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.185+0000 7f8865aca640 1 --2- 172.21.15.3:0/885120812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f884c0052e0 0x7f886013ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.185+0000 7f8865aca640 1 -- 172.21.15.3:0/885120812 >> 172.21.15.3:0/885120812 conn(0x7f88600aab00 msgr2=0x7f88601d2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:33.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.186+0000 7f8865aca640 1 -- 172.21.15.3:0/885120812 shutdown_connections 2024-05-21T17:41:33.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.186+0000 7f8865aca640 1 -- 172.21.15.3:0/885120812 wait complete. 2024-05-21T17:41:33.203 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T17:41:33.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.310+0000 7f3731b6e640 1 Processor -- start 2024-05-21T17:41:33.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.310+0000 7f3731b6e640 1 -- start start 2024-05-21T17:41:33.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f3731b6e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c149680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f3731b6e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f372c06cc60 0x7f372c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f3731b6e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f372c146b80 con 0x7f372c0adbd0 2024-05-21T17:41:33.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f3731b6e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f372c0ac6e0 con 0x7f372c06cc60 2024-05-21T17:41:33.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f3731b6e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f372c0a8300 con 0x7f372c0ac300 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f372b7fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f372bfff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f372c0adbd0 0x7f372c145630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35778/0 (socket says 172.21.15.3:35778) 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f372bfff640 1 -- 172.21.15.3:0/200875753 learned_addr learned my addr 172.21.15.3:0/200875753 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f372affd640 1 --2- 172.21.15.3:0/200875753 >> v2:172.21.15.203:3300/0 conn(0x7f372c06cc60 0x7f372c149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f372a7fc640 1 -- 172.21.15.3:0/200875753 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1851443767 0 0) 0x7f372c146b80 con 0x7f372c0adbd0 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f372b7fe640 1 -- 172.21.15.3:0/200875753 >> v2:172.21.15.203:3300/0 conn(0x7f372c06cc60 msgr2=0x7f372c149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f372b7fe640 1 --2- 172.21.15.3:0/200875753 >> v2:172.21.15.203:3300/0 conn(0x7f372c06cc60 0x7f372c149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f372b7fe640 1 -- 172.21.15.3:0/200875753 >> v1:172.21.15.3:6789/0 conn(0x7f372c0adbd0 legacy=0x7f372c145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.311+0000 7f372b7fe640 1 -- 172.21.15.3:0/200875753 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c0ad450 con 0x7f372c0ac300 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.312+0000 7f372b7fe640 1 --2- 172.21.15.3:0/200875753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c149680 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f371c0097a0 tx=0x7f371c02b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1857e24382304ca2 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.312+0000 7f372a7fc640 1 -- 172.21.15.3:0/200875753 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c038070 con 0x7f372c0ac300 2024-05-21T17:41:33.314 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.312+0000 7f372a7fc640 1 -- 172.21.15.3:0/200875753 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371c031700 con 0x7f372c0ac300 2024-05-21T17:41:33.315 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.312+0000 7f372a7fc640 1 -- 172.21.15.3:0/200875753 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c031ad0 con 0x7f372c0ac300 2024-05-21T17:41:33.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.313+0000 7f3731b6e640 1 -- 172.21.15.3:0/200875753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 msgr2=0x7f372c149680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.313+0000 7f3731b6e640 1 --2- 172.21.15.3:0/200875753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c149680 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f371c0097a0 tx=0x7f371c02b7a0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.313+0000 7f3731b6e640 1 -- 172.21.15.3:0/200875753 shutdown_connections 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.313+0000 7f3731b6e640 1 --2- 172.21.15.3:0/200875753 >> v2:172.21.15.203:3300/0 conn(0x7f372c06cc60 0x7f372c149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.313+0000 7f3731b6e640 1 --2- 172.21.15.3:0/200875753 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c149680 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.313+0000 7f3731b6e640 1 -- 172.21.15.3:0/200875753 >> 172.21.15.3:0/200875753 conn(0x7f372c0aaa90 msgr2=0x7f372c0aaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.313+0000 7f3731b6e640 1 -- 172.21.15.3:0/200875753 shutdown_connections 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.313+0000 7f3731b6e640 1 -- 172.21.15.3:0/200875753 wait complete. 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.314+0000 7f3731b6e640 1 Processor -- start 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.314+0000 7f3731b6e640 1 -- start start 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.314+0000 7f3731b6e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c06cc60 0x7f372c0bc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.314+0000 7f3731b6e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c13e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f3731b6e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372c0adbd0 0x7f372c1d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f3731b6e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f372c14a4d0 con 0x7f372c0adbd0 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f3731b6e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f372c14a350 con 0x7f372c06cc60 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f3731b6e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f372c145d40 con 0x7f372c0ac300 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372affd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c13e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372c0adbd0 0x7f372c1d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372affd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c13e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:39674/0 (socket says 172.21.15.3:39674) 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372affd640 1 -- 172.21.15.3:0/2815629369 learned_addr learned my addr 172.21.15.3:0/2815629369 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372b7fe640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c06cc60 0x7f372c0bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372affd640 1 -- 172.21.15.3:0/2815629369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c06cc60 msgr2=0x7f372c0bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372affd640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c06cc60 0x7f372c0bc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372affd640 1 -- 172.21.15.3:0/2815629369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372c0adbd0 msgr2=0x7f372c1d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372affd640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372c0adbd0 0x7f372c1d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372affd640 1 -- 172.21.15.3:0/2815629369 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c13def0 con 0x7f372c0ac300 2024-05-21T17:41:33.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372bfff640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372c0adbd0 0x7f372c1d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:33.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f372affd640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c13e3a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f372000e9b0 tx=0x7f372000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:33.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f3713fff640 1 -- 172.21.15.3:0/2815629369 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720017070 con 0x7f372c0ac300 2024-05-21T17:41:33.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f3713fff640 1 -- 172.21.15.3:0/2815629369 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f372000c9a0 con 0x7f372c0ac300 2024-05-21T17:41:33.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f3731b6e640 1 -- 172.21.15.3:0/2815629369 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c13e8e0 con 0x7f372c0ac300 2024-05-21T17:41:33.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f3713fff640 1 -- 172.21.15.3:0/2815629369 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372000cd70 con 0x7f372c0ac300 2024-05-21T17:41:33.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.315+0000 7f3731b6e640 1 -- 172.21.15.3:0/2815629369 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1d2150 con 0x7f372c0ac300 2024-05-21T17:41:33.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.316+0000 7f3731b6e640 1 -- 172.21.15.3:0/2815629369 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372c069a90 con 0x7f372c0ac300 2024-05-21T17:41:33.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.318+0000 7f3713fff640 1 -- 172.21.15.3:0/2815629369 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f372001a070 con 0x7f372c0ac300 2024-05-21T17:41:33.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.318+0000 7f3713fff640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:33.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.319+0000 7f3713fff640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:33.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.319+0000 7f3713fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:33.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.319+0000 7f3713fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:33.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.319+0000 7f3713fff640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37180af260 0x7f37180b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.319+0000 7f3713fff640 1 -- 172.21.15.3:0/2815629369 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(15..15 src has 1..15) ==== 4334+0+0 (secure 0 0 0) 0x7f37200dcfd0 con 0x7f372c0ac300 2024-05-21T17:41:33.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.319+0000 7f372b7fe640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37180af260 0x7f37180b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.320+0000 7f372b7fe640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37180af260 0x7f37180b1720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f371c035000 tx=0x7f371c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:33.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.322+0000 7f3713fff640 1 -- 172.21.15.3:0/2815629369 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f37200a6730 con 0x7f372c0ac300 2024-05-21T17:41:33.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.516+0000 7f3731b6e640 1 -- 172.21.15.3:0/2815629369 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f372c06d510 con 0x7f372c0ac300 2024-05-21T17:41:33.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.517+0000 7f3713fff640 1 -- 172.21.15.3:0/2815629369 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+13534 (secure 0 0 0) 0x7f37200aaa30 con 0x7f372c0ac300 2024-05-21T17:41:33.518 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:33.519 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"fsid":"e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442","created":"2024-05-21T17:41:17.034337+0000","modified":"2024-05-21T17:41:32.553131+0000","last_up_change":"2024-05-21T17:41:25.136644+0000","last_in_change":"2024-05-21T17:41:18.179591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T17:41:25.537561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-21T17:41:26.016407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-05-21T17:41:30.560078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-05-21T17:41:31.496059+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7ac7071-219d-4cf4-9f8a-977f495fd0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6801","nonce":3296405141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6803","nonce":3296405141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6807","nonce":3296405141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6805","nonce":3296405141}]},"public_addr":"172.21.15.3:6801/3296405141","cluster_addr":"172.21.15.3:6803/3296405141","heartbeat_back_addr":"172.21.15.3:6807/3296405141","heartbeat_front_addr":"172.21.15.3:6805/3296405141","state":["exists","up"]},{"osd":1,"uuid":"89df5105-5900-4ffa-b8c1-6f480865a085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6801","nonce":1777872358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6803","nonce":1777872358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6810","nonce":1777872358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6805","nonce":1777872358}]},"public_addr":"172.21.15.203:6801/1777872358","cluster_addr":"172.21.15.203:6803/1777872358","heartbeat_back_addr":"172.21.15.203:6810/1777872358","heartbeat_front_addr":"172.21.15.203:6805/1777872358","state":["exists","up"]},{"osd":2,"uuid":"15f83334-8efc-40c2-af0c-eeed45923086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6817","nonce":1069390381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6819","nonce":1069390381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6823","nonce":1069390381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6821","nonce":1069390381}]},"public_addr":"172.21.15.203:6817/1069390381","cluster_addr":"172.21.15.203:6819/1069390381","heartbeat_back_addr":"172.21.15.203:6823/1069390381","heartbeat_front_addr":"172.21.15.203:6821/1069390381","state":["exists","up"]},{"osd":3,"uuid":"6a27ac8e-f957-4455-b477-3090081766c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6808","nonce":1072938612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6811","nonce":1072938612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6815","nonce":1072938612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6813","nonce":1072938612}]},"public_addr":"172.21.15.203:6808/1072938612","cluster_addr":"172.21.15.203:6811/1072938612","heartbeat_back_addr":"172.21.15.203:6815/1072938612","heartbeat_front_addr":"172.21.15.203:6813/1072938612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.224040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.344461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.319792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.332782+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T17:41:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 10 mgrc shutdown 2024-05-21T17:41:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 -- 172.21.15.3:0/2815629369 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37180af260 msgr2=0x7f37180b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37180af260 0x7f37180b1720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f371c035000 tx=0x7f371c008030 comp rx=0 tx=0).stop 2024-05-21T17:41:33.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 -- 172.21.15.3:0/2815629369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 msgr2=0x7f372c13e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c13e3a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f372000e9b0 tx=0x7f372000ee70 comp rx=0 tx=0).stop 2024-05-21T17:41:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 -- 172.21.15.3:0/2815629369 shutdown_connections 2024-05-21T17:41:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37180af260 0x7f37180b1720 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f372c0adbd0 0x7f372c1d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f372c0ac300 0x7f372c13e3a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 --2- 172.21.15.3:0/2815629369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f372c06cc60 0x7f372c0bc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 -- 172.21.15.3:0/2815629369 >> 172.21.15.3:0/2815629369 conn(0x7f372c0aaa90 msgr2=0x7f372c0a9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 -- 172.21.15.3:0/2815629369 shutdown_connections 2024-05-21T17:41:33.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.520+0000 7f3731b6e640 1 -- 172.21.15.3:0/2815629369 wait complete. 2024-05-21T17:41:33.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T17:41:33.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.634+0000 7f018e233640 1 Processor -- start 2024-05-21T17:41:33.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.634+0000 7f018e233640 1 -- start start 2024-05-21T17:41:33.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f018e233640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0188146650 0x7f01880ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f018e233640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0188146b20 con 0x7f01880ac330 2024-05-21T17:41:33.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f018e233640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0188146de0 con 0x7f0188149e30 2024-05-21T17:41:33.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f018e233640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f01880a8530 con 0x7f0188146650 2024-05-21T17:41:33.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f0186ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0188146650 0x7f01880ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f01877fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f01880ac330 0x7f0188149720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35780/0 (socket says 172.21.15.3:35780) 2024-05-21T17:41:33.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f0186ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0188146650 0x7f01880ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50308/0 (socket says 172.21.15.3:50308) 2024-05-21T17:41:33.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f01877fe640 1 -- 172.21.15.3:0/521564087 learned_addr learned my addr 172.21.15.3:0/521564087 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:33.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f0186ffd640 1 -- 172.21.15.3:0/521564087 >> v1:172.21.15.203:6789/0 conn(0x7f0188149e30 legacy=0x7f0188150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f0186ffd640 1 -- 172.21.15.3:0/521564087 >> v1:172.21.15.3:6789/0 conn(0x7f01880ac330 legacy=0x7f0188149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.635+0000 7f0186ffd640 1 -- 172.21.15.3:0/521564087 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01880ac710 con 0x7f0188146650 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.636+0000 7f0186ffd640 1 --2- 172.21.15.3:0/521564087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0188146650 0x7f01880ad480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f017c0097a0 tx=0x7f017c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7cb5d3171b9fca server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.636+0000 7f01867fc640 1 -- 172.21.15.3:0/521564087 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c035020 con 0x7f0188146650 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.636+0000 7f01867fc640 1 -- 172.21.15.3:0/521564087 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f017c031830 con 0x7f0188146650 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.636+0000 7f01867fc640 1 -- 172.21.15.3:0/521564087 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c031c00 con 0x7f0188146650 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.637+0000 7f018e233640 1 -- 172.21.15.3:0/521564087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0188146650 msgr2=0x7f01880ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.637+0000 7f018e233640 1 --2- 172.21.15.3:0/521564087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0188146650 0x7f01880ad480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f017c0097a0 tx=0x7f017c02b8d0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.637+0000 7f018e233640 1 -- 172.21.15.3:0/521564087 shutdown_connections 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.637+0000 7f018e233640 1 --2- 172.21.15.3:0/521564087 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0188146650 0x7f01880ad480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.637+0000 7f018e233640 1 -- 172.21.15.3:0/521564087 >> 172.21.15.3:0/521564087 conn(0x7f01880aab00 msgr2=0x7f01880aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.637+0000 7f018e233640 1 -- 172.21.15.3:0/521564087 shutdown_connections 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.637+0000 7f018e233640 1 -- 172.21.15.3:0/521564087 wait complete. 2024-05-21T17:41:33.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.638+0000 7f018e233640 1 Processor -- start 2024-05-21T17:41:33.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.638+0000 7f018e233640 1 -- start start 2024-05-21T17:41:33.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f018e233640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01880ac330 0x7f01881488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f018e233640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0188146650 0x7f0188149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f018e233640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0188149e30 0x7f0188140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f018e233640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f018806ce70 con 0x7f01880ac330 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f018e233640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f018806ccf0 con 0x7f0188149e30 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f018e233640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f018814a280 con 0x7f0188146650 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f01877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01880ac330 0x7f01881488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f01877fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01880ac330 0x7f01881488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50312/0 (socket says 172.21.15.3:50312) 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f01877fe640 1 -- 172.21.15.3:0/3203578956 learned_addr learned my addr 172.21.15.3:0/3203578956 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f0186ffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0188146650 0x7f0188149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f0187fff640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0188149e30 0x7f0188140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f01877fe640 1 -- 172.21.15.3:0/3203578956 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0188146650 msgr2=0x7f0188149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f01877fe640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0188146650 0x7f0188149130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f01877fe640 1 -- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0188149e30 msgr2=0x7f0188140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f01877fe640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0188149e30 0x7f0188140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f01877fe640 1 -- 172.21.15.3:0/3203578956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018813f040 con 0x7f01880ac330 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f0187fff640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0188149e30 0x7f0188140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f01877fe640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01880ac330 0x7f01881488e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f017800c500 tx=0x7f017800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f016ffff640 1 -- 172.21.15.3:0/3203578956 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178017070 con 0x7f01880ac330 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f016ffff640 1 -- 172.21.15.3:0/3203578956 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0178005ce0 con 0x7f01880ac330 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f018e233640 1 -- 172.21.15.3:0/3203578956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f018813f300 con 0x7f01880ac330 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f016ffff640 1 -- 172.21.15.3:0/3203578956 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01780072f0 con 0x7f01880ac330 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.639+0000 7f018e233640 1 -- 172.21.15.3:0/3203578956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f018813fd70 con 0x7f01880ac330 2024-05-21T17:41:33.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.640+0000 7f018e233640 1 -- 172.21.15.3:0/3203578956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0188140f90 con 0x7f01880ac330 2024-05-21T17:41:33.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.642+0000 7f016ffff640 1 -- 172.21.15.3:0/3203578956 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f017801a070 con 0x7f01880ac330 2024-05-21T17:41:33.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.642+0000 7f016ffff640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:33.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.642+0000 7f016ffff640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:33.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.642+0000 7f016ffff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:33.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.642+0000 7f016ffff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:33.644 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.642+0000 7f016ffff640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f01740af260 0x7f01740b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.643+0000 7f016ffff640 1 -- 172.21.15.3:0/3203578956 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4334+0+0 (secure 0 0 0) 0x7f01780dbab0 con 0x7f01880ac330 2024-05-21T17:41:33.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.643+0000 7f0186ffd640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f01740af260 0x7f01740b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.645 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.643+0000 7f0186ffd640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f01740af260 0x7f01740b1720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f017c036000 tx=0x7f017c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:33.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.647+0000 7f016ffff640 1 -- 172.21.15.3:0/3203578956 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f01780a52a0 con 0x7f01880ac330 2024-05-21T17:41:33.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.837+0000 7f018e233640 1 -- 172.21.15.3:0/3203578956 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0188069a90 con 0x7f01880ac330 2024-05-21T17:41:33.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.838+0000 7f016ffff640 1 -- 172.21.15.3:0/3203578956 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+13534 (secure 0 0 0) 0x7f01780a95a0 con 0x7f01880ac330 2024-05-21T17:41:33.839 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:33.839 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":15,"fsid":"e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442","created":"2024-05-21T17:41:17.034337+0000","modified":"2024-05-21T17:41:32.553131+0000","last_up_change":"2024-05-21T17:41:25.136644+0000","last_in_change":"2024-05-21T17:41:18.179591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T17:41:25.537561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-21T17:41:26.016407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-05-21T17:41:30.560078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-05-21T17:41:31.496059+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7ac7071-219d-4cf4-9f8a-977f495fd0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6801","nonce":3296405141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6803","nonce":3296405141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6807","nonce":3296405141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6805","nonce":3296405141}]},"public_addr":"172.21.15.3:6801/3296405141","cluster_addr":"172.21.15.3:6803/3296405141","heartbeat_back_addr":"172.21.15.3:6807/3296405141","heartbeat_front_addr":"172.21.15.3:6805/3296405141","state":["exists","up"]},{"osd":1,"uuid":"89df5105-5900-4ffa-b8c1-6f480865a085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6801","nonce":1777872358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6803","nonce":1777872358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6810","nonce":1777872358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6805","nonce":1777872358}]},"public_addr":"172.21.15.203:6801/1777872358","cluster_addr":"172.21.15.203:6803/1777872358","heartbeat_back_addr":"172.21.15.203:6810/1777872358","heartbeat_front_addr":"172.21.15.203:6805/1777872358","state":["exists","up"]},{"osd":2,"uuid":"15f83334-8efc-40c2-af0c-eeed45923086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6817","nonce":1069390381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6819","nonce":1069390381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6823","nonce":1069390381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6821","nonce":1069390381}]},"public_addr":"172.21.15.203:6817/1069390381","cluster_addr":"172.21.15.203:6819/1069390381","heartbeat_back_addr":"172.21.15.203:6823/1069390381","heartbeat_front_addr":"172.21.15.203:6821/1069390381","state":["exists","up"]},{"osd":3,"uuid":"6a27ac8e-f957-4455-b477-3090081766c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6808","nonce":1072938612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6811","nonce":1072938612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6815","nonce":1072938612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6813","nonce":1072938612}]},"public_addr":"172.21.15.203:6808/1072938612","cluster_addr":"172.21.15.203:6811/1072938612","heartbeat_back_addr":"172.21.15.203:6815/1072938612","heartbeat_front_addr":"172.21.15.203:6813/1072938612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.224040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.344461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.319792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.332782+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T17:41:33.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 10 mgrc shutdown 2024-05-21T17:41:33.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 -- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f01740af260 msgr2=0x7f01740b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f01740af260 0x7f01740b1720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f017c036000 tx=0x7f017c008030 comp rx=0 tx=0).stop 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 -- 172.21.15.3:0/3203578956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01880ac330 msgr2=0x7f01881488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01880ac330 0x7f01881488e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f017800c500 tx=0x7f017800c9c0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 -- 172.21.15.3:0/3203578956 shutdown_connections 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f01740af260 0x7f01740b1720 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0188149e30 0x7f0188140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0188146650 0x7f0188149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 --2- 172.21.15.3:0/3203578956 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01880ac330 0x7f01881488e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 -- 172.21.15.3:0/3203578956 >> 172.21.15.3:0/3203578956 conn(0x7f01880aab00 msgr2=0x7f01880bc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 -- 172.21.15.3:0/3203578956 shutdown_connections 2024-05-21T17:41:33.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.840+0000 7f018e233640 1 -- 172.21.15.3:0/3203578956 wait complete. 2024-05-21T17:41:33.852 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-21T17:41:33.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.957+0000 7f234e0ae640 1 Processor -- start 2024-05-21T17:41:33.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.957+0000 7f234e0ae640 1 -- start start 2024-05-21T17:41:33.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f234e0ae640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f23481387f0 0x7f23480ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f234e0ae640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 0x7f2348150b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f234e0ae640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2348138c80 con 0x7f23480abf70 2024-05-21T17:41:33.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f234e0ae640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2348139810 con 0x7f23481387f0 2024-05-21T17:41:33.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f234e0ae640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f2348138f20 con 0x7f23480ad6d0 2024-05-21T17:41:33.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f2346ffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f23481387f0 0x7f23480ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f2347fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 0x7f2348150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f23477fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f23480abf70 0x7f234806c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:35792/0 (socket says 172.21.15.3:35792) 2024-05-21T17:41:33.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f23477fe640 1 -- 172.21.15.3:0/2340695201 learned_addr learned my addr 172.21.15.3:0/2340695201 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:33.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f2347fff640 1 -- 172.21.15.3:0/2340695201 >> v2:172.21.15.3:3300/0 conn(0x7f23481387f0 msgr2=0x7f23480ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:33.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f2347fff640 1 --2- 172.21.15.3:0/2340695201 >> v2:172.21.15.3:3300/0 conn(0x7f23481387f0 0x7f23480ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f2347fff640 1 -- 172.21.15.3:0/2340695201 >> v1:172.21.15.3:6789/0 conn(0x7f23480abf70 legacy=0x7f234806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f2347fff640 1 -- 172.21.15.3:0/2340695201 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234813c970 con 0x7f23480ad6d0 2024-05-21T17:41:33.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f23467fc640 1 -- 172.21.15.3:0/2340695201 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1891247659 0 0) 0x7f2348138c80 con 0x7f23480abf70 2024-05-21T17:41:33.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.958+0000 7f2346ffd640 1 --2- 172.21.15.3:0/2340695201 >> v2:172.21.15.3:3300/0 conn(0x7f23481387f0 0x7f23480ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:33.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.959+0000 7f2347fff640 1 --2- 172.21.15.3:0/2340695201 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 0x7f2348150b20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2340009d90 tx=0x7f234002be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ff877e7032f98df server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:33.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.959+0000 7f23467fc640 1 -- 172.21.15.3:0/2340695201 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340039070 con 0x7f23480ad6d0 2024-05-21T17:41:33.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.959+0000 7f23467fc640 1 -- 172.21.15.3:0/2340695201 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234002cbd0 con 0x7f23480ad6d0 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.959+0000 7f23467fc640 1 -- 172.21.15.3:0/2340695201 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340034440 con 0x7f23480ad6d0 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.960+0000 7f234e0ae640 1 -- 172.21.15.3:0/2340695201 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 msgr2=0x7f2348150b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.960+0000 7f234e0ae640 1 --2- 172.21.15.3:0/2340695201 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 0x7f2348150b20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2340009d90 tx=0x7f234002be20 comp rx=0 tx=0).stop 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.960+0000 7f234e0ae640 1 -- 172.21.15.3:0/2340695201 shutdown_connections 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.960+0000 7f234e0ae640 1 --2- 172.21.15.3:0/2340695201 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 0x7f2348150b20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.960+0000 7f234e0ae640 1 --2- 172.21.15.3:0/2340695201 >> v2:172.21.15.3:3300/0 conn(0x7f23481387f0 0x7f23480ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.960+0000 7f234e0ae640 1 -- 172.21.15.3:0/2340695201 >> 172.21.15.3:0/2340695201 conn(0x7f23480aab70 msgr2=0x7f23480aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.960+0000 7f234e0ae640 1 -- 172.21.15.3:0/2340695201 shutdown_connections 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.960+0000 7f234e0ae640 1 -- 172.21.15.3:0/2340695201 wait complete. 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.961+0000 7f234e0ae640 1 Processor -- start 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.961+0000 7f234e0ae640 1 -- start start 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f234e0ae640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23480abf70 0x7f23480a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f234e0ae640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 0x7f2348147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f234e0ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23480b5a30 0x7f2348134860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f234e0ae640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f23480adae0 con 0x7f23480abf70 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f234e0ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f23480a52e0 con 0x7f23480b5a30 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f234e0ae640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f23480adc60 con 0x7f23480ad6d0 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f23477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23480abf70 0x7f23480a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f2346ffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 0x7f2348147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f23477fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23480abf70 0x7f23480a7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:50320/0 (socket says 172.21.15.3:50320) 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f23477fe640 1 -- 172.21.15.3:0/2052822236 learned_addr learned my addr 172.21.15.3:0/2052822236 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f2347fff640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23480b5a30 0x7f2348134860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f23477fe640 1 -- 172.21.15.3:0/2052822236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 msgr2=0x7f2348147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f23477fe640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 0x7f2348147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f23477fe640 1 -- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23480b5a30 msgr2=0x7f2348134860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f23477fe640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23480b5a30 0x7f2348134860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:33.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f23477fe640 1 -- 172.21.15.3:0/2052822236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2348135a40 con 0x7f23480abf70 2024-05-21T17:41:33.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f2347fff640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23480b5a30 0x7f2348134860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:33.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f2346ffd640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 0x7f2348147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:33.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f23477fe640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23480abf70 0x7f23480a7800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f233800c490 tx=0x7f233800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:33.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.962+0000 7f232ffff640 1 -- 172.21.15.3:0/2052822236 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338018070 con 0x7f23480abf70 2024-05-21T17:41:33.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.963+0000 7f232ffff640 1 -- 172.21.15.3:0/2052822236 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2338007500 con 0x7f23480abf70 2024-05-21T17:41:33.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.963+0000 7f234e0ae640 1 -- 172.21.15.3:0/2052822236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23481481d0 con 0x7f23480abf70 2024-05-21T17:41:33.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.963+0000 7f232ffff640 1 -- 172.21.15.3:0/2052822236 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338004a20 con 0x7f23480abf70 2024-05-21T17:41:33.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.963+0000 7f234e0ae640 1 -- 172.21.15.3:0/2052822236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23481325e0 con 0x7f23480abf70 2024-05-21T17:41:33.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.963+0000 7f234e0ae640 1 -- 172.21.15.3:0/2052822236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2348069a90 con 0x7f23480abf70 2024-05-21T17:41:33.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.965+0000 7f232ffff640 1 -- 172.21.15.3:0/2052822236 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f233801b070 con 0x7f23480abf70 2024-05-21T17:41:33.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.965+0000 7f232ffff640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:33.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.966+0000 7f232ffff640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:33.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.966+0000 7f232ffff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:33.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.966+0000 7f232ffff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:33.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.966+0000 7f232ffff640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f23340af260 0x7f23340b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:33.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.966+0000 7f232ffff640 1 -- 172.21.15.3:0/2052822236 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4334+0+0 (secure 0 0 0) 0x7f23380dbf30 con 0x7f23480abf70 2024-05-21T17:41:33.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.966+0000 7f2346ffd640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f23340af260 0x7f23340b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:33.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.966+0000 7f2346ffd640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f23340af260 0x7f23340b1720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f233c009560 tx=0x7f233c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:33.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:33.969+0000 7f232ffff640 1 -- 172.21.15.3:0/2052822236 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f23380a5c40 con 0x7f23480abf70 2024-05-21T17:41:34.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:34.201+0000 7f234e0ae640 1 -- 172.21.15.3:0/2052822236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f234806d540 con 0x7f23480abf70 2024-05-21T17:41:35.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.204+0000 7f232ffff640 1 -- 172.21.15.3:0/2052822236 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f23380a5540 con 0x7f23480abf70 2024-05-21T17:41:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.207+0000 7f234e0ae640 10 mgrc shutdown 2024-05-21T17:41:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.207+0000 7f234e0ae640 1 -- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f23340af260 msgr2=0x7f23340b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.207+0000 7f234e0ae640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f23340af260 0x7f23340b1720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f233c009560 tx=0x7f233c009190 comp rx=0 tx=0).stop 2024-05-21T17:41:35.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.207+0000 7f234e0ae640 1 -- 172.21.15.3:0/2052822236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23480abf70 msgr2=0x7f23480a7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.208+0000 7f234e0ae640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23480abf70 0x7f23480a7800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f233800c490 tx=0x7f233800c950 comp rx=0 tx=0).stop 2024-05-21T17:41:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.208+0000 7f234e0ae640 1 -- 172.21.15.3:0/2052822236 shutdown_connections 2024-05-21T17:41:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.208+0000 7f234e0ae640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f23340af260 0x7f23340b1720 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.208+0000 7f234e0ae640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23480b5a30 0x7f2348134860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.208+0000 7f234e0ae640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f23480ad6d0 0x7f2348147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.208+0000 7f234e0ae640 1 --2- 172.21.15.3:0/2052822236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f23480abf70 0x7f23480a7800 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.208+0000 7f234e0ae640 1 -- 172.21.15.3:0/2052822236 >> 172.21.15.3:0/2052822236 conn(0x7f23480aab70 msgr2=0x7f23480a8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.208+0000 7f234e0ae640 1 -- 172.21.15.3:0/2052822236 shutdown_connections 2024-05-21T17:41:35.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.208+0000 7f234e0ae640 1 -- 172.21.15.3:0/2052822236 wait complete. 2024-05-21T17:41:35.222 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-21T17:41:35.222 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-21T17:41:35.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.334+0000 7f4a1012b640 1 Processor -- start 2024-05-21T17:41:35.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.334+0000 7f4a1012b640 1 -- start start 2024-05-21T17:41:35.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.334+0000 7f4a1012b640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a080b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:35.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a1012b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a080a5850 0x7f4a080a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:35.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a1012b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a080a7290 con 0x7f4a080a61b0 2024-05-21T17:41:35.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a1012b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f4a080a7550 con 0x7f4a080a5850 2024-05-21T17:41:35.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a1012b640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f4a080b90c0 con 0x7f4a08145b60 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0dec7640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a080b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0d6c6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a080a5850 0x7f4a080a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0d6c6640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f4a080a5850 0x7f4a080a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51316/0 (socket says 172.21.15.3:51316) 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0d6c6640 1 -- 172.21.15.3:0/2233224732 learned_addr learned my addr 172.21.15.3:0/2233224732 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0e6c8640 1 --1- 172.21.15.3:0/2233224732 >> v1:172.21.15.3:6789/0 conn(0x7f4a080a61b0 0x7f4a0806c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60644/0 (socket says 172.21.15.3:60644) 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0d6c6640 1 -- 172.21.15.3:0/2233224732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 msgr2=0x7f4a080b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0d6c6640 1 --2- 172.21.15.3:0/2233224732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a080b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0d6c6640 1 -- 172.21.15.3:0/2233224732 >> v1:172.21.15.3:6789/0 conn(0x7f4a080a61b0 legacy=0x7f4a0806c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0cec5640 1 -- 172.21.15.3:0/2233224732 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 893378087 0 0) 0x7f4a080a7290 con 0x7f4a080a61b0 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0d6c6640 1 -- 172.21.15.3:0/2233224732 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a08145f40 con 0x7f4a080a5850 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.335+0000 7f4a0dec7640 1 --2- 172.21.15.3:0/2233224732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a080b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.336+0000 7f4a0d6c6640 1 --2- 172.21.15.3:0/2233224732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a5850 0x7f4a080a5c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009870 tx=0x7f49f802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fdd0b51b2d5bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:35.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.336+0000 7f4a0cec5640 1 -- 172.21.15.3:0/2233224732 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f80317c0 con 0x7f4a080a5850 2024-05-21T17:41:35.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.336+0000 7f4a0cec5640 1 -- 172.21.15.3:0/2233224732 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f8037070 con 0x7f4a080a5850 2024-05-21T17:41:35.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.336+0000 7f4a0cec5640 1 -- 172.21.15.3:0/2233224732 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8031b80 con 0x7f4a080a5850 2024-05-21T17:41:35.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.336+0000 7f4a1012b640 1 -- 172.21.15.3:0/2233224732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a5850 msgr2=0x7f4a080a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:35.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.336+0000 7f4a1012b640 1 --2- 172.21.15.3:0/2233224732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a5850 0x7f4a080a5c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009870 tx=0x7f49f802b860 comp rx=0 tx=0).stop 2024-05-21T17:41:35.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.337+0000 7f4a1012b640 1 -- 172.21.15.3:0/2233224732 shutdown_connections 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.337+0000 7f4a1012b640 1 --2- 172.21.15.3:0/2233224732 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a5850 0x7f4a080a5c70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.337+0000 7f4a1012b640 1 --2- 172.21.15.3:0/2233224732 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a080b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.337+0000 7f4a1012b640 1 -- 172.21.15.3:0/2233224732 >> 172.21.15.3:0/2233224732 conn(0x7f4a0806d700 msgr2=0x7f4a0806db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.337+0000 7f4a1012b640 1 -- 172.21.15.3:0/2233224732 shutdown_connections 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.337+0000 7f4a1012b640 1 -- 172.21.15.3:0/2233224732 wait complete. 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a1012b640 1 Processor -- start 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a1012b640 1 -- start start 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a1012b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a080a5850 0x7f4a08147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a1012b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a61b0 0x7f4a080a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a1012b640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a081413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:35.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a1012b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08070420 con 0x7f4a080a61b0 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a1012b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080702a0 con 0x7f4a080a5850 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a1012b640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f4a08071db0 con 0x7f4a08145b60 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a0e6c8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a081413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a0d6c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a61b0 0x7f4a080a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a0d6c6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a61b0 0x7f4a080a41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51326/0 (socket says 172.21.15.3:51326) 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a0d6c6640 1 -- 172.21.15.3:0/3962361135 learned_addr learned my addr 172.21.15.3:0/3962361135 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.338+0000 7f4a0e6c8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a081413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40774/0 (socket says 172.21.15.3:40774) 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f4a0dec7640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a080a5850 0x7f4a08147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f4a0d6c6640 1 -- 172.21.15.3:0/3962361135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 msgr2=0x7f4a081413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f4a0d6c6640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a081413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f4a0d6c6640 1 -- 172.21.15.3:0/3962361135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a080a5850 msgr2=0x7f4a08147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f4a0d6c6640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a080a5850 0x7f4a08147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:35.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f4a0d6c6640 1 -- 172.21.15.3:0/3962361135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a08148380 con 0x7f4a080a61b0 2024-05-21T17:41:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f4a0e6c8640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a081413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f4a0d6c6640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a61b0 0x7f4a080a41f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49f80090b0 tx=0x7f49f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f49fe7fc640 1 -- 172.21.15.3:0/3962361135 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f804a070 con 0x7f4a080a61b0 2024-05-21T17:41:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f49fe7fc640 1 -- 172.21.15.3:0/3962361135 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f8032450 con 0x7f4a080a61b0 2024-05-21T17:41:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f4a1012b640 1 -- 172.21.15.3:0/3962361135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a08148b20 con 0x7f4a080a61b0 2024-05-21T17:41:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f49fe7fc640 1 -- 172.21.15.3:0/3962361135 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8032840 con 0x7f4a080a61b0 2024-05-21T17:41:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.339+0000 7f4a1012b640 1 -- 172.21.15.3:0/3962361135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a08148810 con 0x7f4a080a61b0 2024-05-21T17:41:35.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.340+0000 7f4a1012b640 1 -- 172.21.15.3:0/3962361135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a081497e0 con 0x7f4a080a61b0 2024-05-21T17:41:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.341+0000 7f49fe7fc640 1 -- 172.21.15.3:0/3962361135 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f49f804d070 con 0x7f4a080a61b0 2024-05-21T17:41:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.341+0000 7f49fe7fc640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.342+0000 7f49fe7fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.342+0000 7f49fe7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.342+0000 7f49fe7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.342+0000 7f49fe7fc640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f49ec0af260 0x7f49ec0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.342+0000 7f49fe7fc640 1 -- 172.21.15.3:0/3962361135 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f49f80fe910 con 0x7f4a080a61b0 2024-05-21T17:41:35.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.342+0000 7f4a0dec7640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f49ec0af260 0x7f49ec0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:35.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.343+0000 7f4a0dec7640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f49ec0af260 0x7f49ec0b1720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009950 tx=0x7f4a04008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:35.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.345+0000 7f49fe7fc640 1 -- 172.21.15.3:0/3962361135 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f49f8054020 con 0x7f4a080a61b0 2024-05-21T17:41:35.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:35.578+0000 7f4a1012b640 1 -- 172.21.15.3:0/3962361135 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4a08144e30 con 0x7f4a080a61b0 2024-05-21T17:41:36.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.209+0000 7f49fe7fc640 1 -- 172.21.15.3:0/3962361135 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f49f8040070 con 0x7f4a080a61b0 2024-05-21T17:41:36.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 10 mgrc shutdown 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 -- 172.21.15.3:0/3962361135 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f49ec0af260 msgr2=0x7f49ec0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f49ec0af260 0x7f49ec0b1720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009950 tx=0x7f4a04008040 comp rx=0 tx=0).stop 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 -- 172.21.15.3:0/3962361135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a61b0 msgr2=0x7f4a080a41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a61b0 0x7f4a080a41f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49f80090b0 tx=0x7f49f80046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 -- 172.21.15.3:0/3962361135 shutdown_connections 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f49ec0af260 0x7f49ec0b1720 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4a08145b60 0x7f4a081413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a080a61b0 0x7f4a080a41f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 --2- 172.21.15.3:0/3962361135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a080a5850 0x7f4a08147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 -- 172.21.15.3:0/3962361135 >> 172.21.15.3:0/3962361135 conn(0x7f4a0806d700 msgr2=0x7f4a080a50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 -- 172.21.15.3:0/3962361135 shutdown_connections 2024-05-21T17:41:36.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.212+0000 7f4a1012b640 1 -- 172.21.15.3:0/3962361135 wait complete. 2024-05-21T17:41:36.226 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-21T17:41:36.227 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T17:41:36.334 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.333+0000 7f265f506640 1 Processor -- start 2024-05-21T17:41:36.335 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.334+0000 7f265f506640 1 -- start start 2024-05-21T17:41:36.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.334+0000 7f265f506640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2658074370 0x7f26581339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.334+0000 7f265f506640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26580ab040 0x7f26580ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.334+0000 7f265f506640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26580a9a10 con 0x7f265813a660 2024-05-21T17:41:36.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.334+0000 7f265f506640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f2658074750 con 0x7f26580ab040 2024-05-21T17:41:36.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.334+0000 7f265f506640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f265813a330 con 0x7f2658074370 2024-05-21T17:41:36.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.334+0000 7f265d2a2640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2658074370 0x7f26581339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.334+0000 7f265caa1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f26580ab040 0x7f26580ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.334+0000 7f265d2a2640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2658074370 0x7f26581339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40778/0 (socket says 172.21.15.3:40778) 2024-05-21T17:41:36.336 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.334+0000 7f265d2a2640 1 -- 172.21.15.3:0/3075837924 learned_addr learned my addr 172.21.15.3:0/3075837924 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:36.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.335+0000 7f265d2a2640 1 -- 172.21.15.3:0/3075837924 >> v2:172.21.15.3:3300/0 conn(0x7f26580ab040 msgr2=0x7f26580ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:36.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.335+0000 7f265d2a2640 1 --2- 172.21.15.3:0/3075837924 >> v2:172.21.15.3:3300/0 conn(0x7f26580ab040 0x7f26580ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.335+0000 7f265d2a2640 1 -- 172.21.15.3:0/3075837924 >> v1:172.21.15.3:6789/0 conn(0x7f265813a660 legacy=0x7f265806c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.335+0000 7f265d2a2640 1 -- 172.21.15.3:0/3075837924 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2658136660 con 0x7f2658074370 2024-05-21T17:41:36.337 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.335+0000 7f265caa1640 1 --2- 172.21.15.3:0/3075837924 >> v2:172.21.15.3:3300/0 conn(0x7f26580ab040 0x7f26580ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.335+0000 7f265d2a2640 1 --2- 172.21.15.3:0/3075837924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2658074370 0x7f26581339f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2648009e50 tx=0x7f264802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f19f54f6608c7cbc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f2657fff640 1 -- 172.21.15.3:0/3075837924 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264802ca70 con 0x7f2658074370 2024-05-21T17:41:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f2657fff640 1 -- 172.21.15.3:0/3075837924 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264802cc10 con 0x7f2658074370 2024-05-21T17:41:36.338 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f2657fff640 1 -- 172.21.15.3:0/3075837924 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480343f0 con 0x7f2658074370 2024-05-21T17:41:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f265f506640 1 -- 172.21.15.3:0/3075837924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2658074370 msgr2=0x7f26581339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f265f506640 1 --2- 172.21.15.3:0/3075837924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2658074370 0x7f26581339f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2648009e50 tx=0x7f264802bed0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f265f506640 1 -- 172.21.15.3:0/3075837924 shutdown_connections 2024-05-21T17:41:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f265f506640 1 --2- 172.21.15.3:0/3075837924 >> v2:172.21.15.3:3300/0 conn(0x7f26580ab040 0x7f26580ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f265f506640 1 --2- 172.21.15.3:0/3075837924 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2658074370 0x7f26581339f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f265f506640 1 -- 172.21.15.3:0/3075837924 >> 172.21.15.3:0/3075837924 conn(0x7f26580713c0 msgr2=0x7f265806d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f265f506640 1 -- 172.21.15.3:0/3075837924 shutdown_connections 2024-05-21T17:41:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.336+0000 7f265f506640 1 -- 172.21.15.3:0/3075837924 wait complete. 2024-05-21T17:41:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.337+0000 7f265f506640 1 Processor -- start 2024-05-21T17:41:36.339 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.337+0000 7f265f506640 1 -- start start 2024-05-21T17:41:36.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265f506640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2658074370 0x7f26581ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265f506640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26580ab040 0x7f26580b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265f506640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f265813a660 0x7f265812a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265f506640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f265813a040 con 0x7f2658074370 2024-05-21T17:41:36.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265f506640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2658136c00 con 0x7f26580ab040 2024-05-21T17:41:36.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265f506640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f2658133f30 con 0x7f265813a660 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265d2a2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2658074370 0x7f26581ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265daa3640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f265813a660 0x7f265812a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265d2a2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2658074370 0x7f26581ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51344/0 (socket says 172.21.15.3:51344) 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265d2a2640 1 -- 172.21.15.3:0/1648603982 learned_addr learned my addr 172.21.15.3:0/1648603982 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265caa1640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26580ab040 0x7f26580b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265d2a2640 1 -- 172.21.15.3:0/1648603982 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f265813a660 msgr2=0x7f265812a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265d2a2640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f265813a660 0x7f265812a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265d2a2640 1 -- 172.21.15.3:0/1648603982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26580ab040 msgr2=0x7f26580b8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265d2a2640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26580ab040 0x7f26580b8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265d2a2640 1 -- 172.21.15.3:0/1648603982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26580b9730 con 0x7f2658074370 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.338+0000 7f265daa3640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f265813a660 0x7f265812a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.339+0000 7f265d2a2640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2658074370 0x7f26581ce1f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2648009f70 tx=0x7f2648004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:36.342 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.339+0000 7f2655ffb640 1 -- 172.21.15.3:0/1648603982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648043020 con 0x7f2658074370 2024-05-21T17:41:36.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.339+0000 7f2655ffb640 1 -- 172.21.15.3:0/1648603982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264803e450 con 0x7f2658074370 2024-05-21T17:41:36.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.339+0000 7f265f506640 1 -- 172.21.15.3:0/1648603982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265812aa70 con 0x7f2658074370 2024-05-21T17:41:36.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.339+0000 7f2655ffb640 1 -- 172.21.15.3:0/1648603982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648009510 con 0x7f2658074370 2024-05-21T17:41:36.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.339+0000 7f265f506640 1 -- 172.21.15.3:0/1648603982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581ceb00 con 0x7f2658074370 2024-05-21T17:41:36.343 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.340+0000 7f265f506640 1 -- 172.21.15.3:0/1648603982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26581ce760 con 0x7f2658074370 2024-05-21T17:41:36.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.341+0000 7f2655ffb640 1 -- 172.21.15.3:0/1648603982 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f2648046020 con 0x7f2658074370 2024-05-21T17:41:36.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.341+0000 7f2655ffb640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:36.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.342+0000 7f2655ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:36.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.342+0000 7f2655ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:36.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.342+0000 7f2655ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:36.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.342+0000 7f2655ffb640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f26440af290 0x7f26440b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.342+0000 7f2655ffb640 1 -- 172.21.15.3:0/1648603982 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f26480ff570 con 0x7f2658074370 2024-05-21T17:41:36.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.342+0000 7f265caa1640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f26440af290 0x7f26440b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.344 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.343+0000 7f265caa1640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f26440af290 0x7f26440b1750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f264c009560 tx=0x7f264c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:36.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.345+0000 7f2655ffb640 1 -- 172.21.15.3:0/1648603982 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f264804d030 con 0x7f2658074370 2024-05-21T17:41:36.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.572+0000 7f265f506640 1 -- 172.21.15.3:0/1648603982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2658069290 con 0x7f2658074370 2024-05-21T17:41:36.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.573+0000 7f2655ffb640 1 -- 172.21.15.3:0/1648603982 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+4345 (secure 0 0 0) 0x7f26480c9280 con 0x7f2658074370 2024-05-21T17:41:36.574 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:36.575 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/3741447020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3741447020},{"type":"v1","addr":"172.21.15.203:6827","nonce":3741447020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6810/205822275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":205822275},{"type":"v1","addr":"172.21.15.3:6810","nonce":205822275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-21T17:41:32.553094+0000","modified":"2024-05-21T17:41:35.579599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6811/1903526925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6809","nonce":1903526925},{"type":"v1","addr":"172.21.15.3:6811","nonce":1903526925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-05-21T17:41:36.575 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-21T17:41:36.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 10 mgrc shutdown 2024-05-21T17:41:36.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 -- 172.21.15.3:0/1648603982 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f26440af290 msgr2=0x7f26440b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f26440af290 0x7f26440b1750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f264c009560 tx=0x7f264c008040 comp rx=0 tx=0).stop 2024-05-21T17:41:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 -- 172.21.15.3:0/1648603982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2658074370 msgr2=0x7f26581ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2658074370 0x7f26581ce1f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2648009f70 tx=0x7f2648004a40 comp rx=0 tx=0).stop 2024-05-21T17:41:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 -- 172.21.15.3:0/1648603982 shutdown_connections 2024-05-21T17:41:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f26440af290 0x7f26440b1750 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f265813a660 0x7f265812a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26580ab040 0x7f26580b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 --2- 172.21.15.3:0/1648603982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2658074370 0x7f26581ce1f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 -- 172.21.15.3:0/1648603982 >> 172.21.15.3:0/1648603982 conn(0x7f26580713c0 msgr2=0x7f2658131df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 -- 172.21.15.3:0/1648603982 shutdown_connections 2024-05-21T17:41:36.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.575+0000 7f265f506640 1 -- 172.21.15.3:0/1648603982 wait complete. 2024-05-21T17:41:36.587 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T17:41:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.689+0000 7ff9e56c3640 1 Processor -- start 2024-05-21T17:41:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.689+0000 7ff9e56c3640 1 -- start start 2024-05-21T17:41:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.690+0000 7ff9e56c3640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9e0144cb0 0x7ff9e0073e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.690+0000 7ff9e56c3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff9e00a6d10 0x7ff9e01476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.690+0000 7ff9e56c3640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e00a7990 con 0x7ff9e00a7180 2024-05-21T17:41:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.690+0000 7ff9e56c3640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff9e01440d0 con 0x7ff9e0144cb0 2024-05-21T17:41:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.690+0000 7ff9e56c3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff9e00a6b00 con 0x7ff9e00a6d10 2024-05-21T17:41:36.692 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.690+0000 7ff9deffd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff9e0144cb0 0x7ff9e0073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.690+0000 7ff9df7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff9e00a7180 0x7ff9e0147bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60662/0 (socket says 172.21.15.3:60662) 2024-05-21T17:41:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.690+0000 7ff9df7fe640 1 -- 172.21.15.3:0/2191213950 learned_addr learned my addr 172.21.15.3:0/2191213950 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.690+0000 7ff9de7fc640 1 --2- 172.21.15.3:0/2191213950 >> v2:172.21.15.203:3300/0 conn(0x7ff9e00a6d10 0x7ff9e01476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.691+0000 7ff9deffd640 1 -- 172.21.15.3:0/2191213950 >> v2:172.21.15.203:3300/0 conn(0x7ff9e00a6d10 msgr2=0x7ff9e01476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.691+0000 7ff9deffd640 1 --2- 172.21.15.3:0/2191213950 >> v2:172.21.15.203:3300/0 conn(0x7ff9e00a6d10 0x7ff9e01476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.691+0000 7ff9deffd640 1 -- 172.21.15.3:0/2191213950 >> v1:172.21.15.3:6789/0 conn(0x7ff9e00a7180 legacy=0x7ff9e0147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.691+0000 7ff9deffd640 1 -- 172.21.15.3:0/2191213950 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e0145090 con 0x7ff9e0144cb0 2024-05-21T17:41:36.693 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.691+0000 7ff9ddffb640 1 -- 172.21.15.3:0/2191213950 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2534151460 0 0) 0x7ff9e00a7990 con 0x7ff9e00a7180 2024-05-21T17:41:36.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.691+0000 7ff9deffd640 1 --2- 172.21.15.3:0/2191213950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0144cb0 0x7ff9e0073e40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0071db0 tx=0x7ff9d002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c63c351020569c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:36.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9ddffb640 1 -- 172.21.15.3:0/2191213950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0039070 con 0x7ff9e0144cb0 2024-05-21T17:41:36.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9ddffb640 1 -- 172.21.15.3:0/2191213950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d002cbf0 con 0x7ff9e0144cb0 2024-05-21T17:41:36.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9ddffb640 1 -- 172.21.15.3:0/2191213950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0034440 con 0x7ff9e0144cb0 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9e56c3640 1 -- 172.21.15.3:0/2191213950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0144cb0 msgr2=0x7ff9e0073e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9e56c3640 1 --2- 172.21.15.3:0/2191213950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0144cb0 0x7ff9e0073e40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0071db0 tx=0x7ff9d002be40 comp rx=0 tx=0).stop 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9e56c3640 1 -- 172.21.15.3:0/2191213950 shutdown_connections 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9e56c3640 1 --2- 172.21.15.3:0/2191213950 >> v2:172.21.15.203:3300/0 conn(0x7ff9e00a6d10 0x7ff9e01476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9e56c3640 1 --2- 172.21.15.3:0/2191213950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e0144cb0 0x7ff9e0073e40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9e56c3640 1 -- 172.21.15.3:0/2191213950 >> 172.21.15.3:0/2191213950 conn(0x7ff9e00a8c40 msgr2=0x7ff9e00a9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9e56c3640 1 -- 172.21.15.3:0/2191213950 shutdown_connections 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.692+0000 7ff9e56c3640 1 -- 172.21.15.3:0/2191213950 wait complete. 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.693+0000 7ff9e56c3640 1 Processor -- start 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.693+0000 7ff9e56c3640 1 -- start start 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9e56c3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9e00a6d10 0x7ff9e00a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9e56c3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e00a7180 0x7ff9e00a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9e56c3640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9e0144cb0 0x7ff9e0140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9e56c3640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e00739c0 con 0x7ff9e00a7180 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9e56c3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0144360 con 0x7ff9e00a6d10 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9e56c3640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e00749c0 con 0x7ff9e0144cb0 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9de7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e00a7180 0x7ff9e00a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9e0144cb0 0x7ff9e0140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9de7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e00a7180 0x7ff9e00a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51364/0 (socket says 172.21.15.3:51364) 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9de7fc640 1 -- 172.21.15.3:0/856339542 learned_addr learned my addr 172.21.15.3:0/856339542 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9deffd640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9e00a6d10 0x7ff9e00a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9df7fe640 1 -- 172.21.15.3:0/856339542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9e00a6d10 msgr2=0x7ff9e00a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9df7fe640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9e00a6d10 0x7ff9e00a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9df7fe640 1 -- 172.21.15.3:0/856339542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e00a7180 msgr2=0x7ff9e00a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9df7fe640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e00a7180 0x7ff9e00a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9df7fe640 1 -- 172.21.15.3:0/856339542 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e01d0780 con 0x7ff9e0144cb0 2024-05-21T17:41:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.694+0000 7ff9de7fc640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e00a7180 0x7ff9e00a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.695+0000 7ff9df7fe640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9e0144cb0 0x7ff9e0140450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800c370 tx=0x7ff9d800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.695+0000 7ff9c77fe640 1 -- 172.21.15.3:0/856339542 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80074e0 con 0x7ff9e0144cb0 2024-05-21T17:41:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.695+0000 7ff9c77fe640 1 -- 172.21.15.3:0/856339542 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d80047e0 con 0x7ff9e0144cb0 2024-05-21T17:41:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.695+0000 7ff9e56c3640 1 -- 172.21.15.3:0/856339542 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e01d0a40 con 0x7ff9e0144cb0 2024-05-21T17:41:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.695+0000 7ff9c77fe640 1 -- 172.21.15.3:0/856339542 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8004bd0 con 0x7ff9e0144cb0 2024-05-21T17:41:36.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.695+0000 7ff9e56c3640 1 -- 172.21.15.3:0/856339542 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e01433e0 con 0x7ff9e0144cb0 2024-05-21T17:41:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.696+0000 7ff9e56c3640 1 -- 172.21.15.3:0/856339542 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e0069290 con 0x7ff9e0144cb0 2024-05-21T17:41:36.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.697+0000 7ff9c77fe640 1 -- 172.21.15.3:0/856339542 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7ff9d8011070 con 0x7ff9e0144cb0 2024-05-21T17:41:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.697+0000 7ff9c77fe640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.698+0000 7ff9c77fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.698+0000 7ff9c77fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.698+0000 7ff9c77fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.698+0000 7ff9c77fe640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff9cc0af260 0x7ff9cc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.698+0000 7ff9c77fe640 1 -- 172.21.15.3:0/856339542 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7ff9d80dc320 con 0x7ff9e0144cb0 2024-05-21T17:41:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.698+0000 7ff9deffd640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff9cc0af260 0x7ff9cc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:36.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.699+0000 7ff9deffd640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff9cc0af260 0x7ff9cc0b1720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0036000 tx=0x7ff9d0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:36.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.702+0000 7ff9c77fe640 1 -- 172.21.15.3:0/856339542 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff9d80a6030 con 0x7ff9e0144cb0 2024-05-21T17:41:36.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.896+0000 7ff9e56c3640 1 -- 172.21.15.3:0/856339542 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9e0073c20 con 0x7ff9e0144cb0 2024-05-21T17:41:36.898 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.897+0000 7ff9c77fe640 1 -- 172.21.15.3:0/856339542 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+13521 (secure 0 0 0) 0x7ff9d80ae0b0 con 0x7ff9e0144cb0 2024-05-21T17:41:36.901 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:36.901 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442","created":"2024-05-21T17:41:17.034337+0000","modified":"2024-05-21T17:41:34.184816+0000","last_up_change":"2024-05-21T17:41:25.136644+0000","last_in_change":"2024-05-21T17:41:18.179591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T17:41:25.537561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-21T17:41:26.016407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-05-21T17:41:30.560078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-05-21T17:41:31.496059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7ac7071-219d-4cf4-9f8a-977f495fd0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6801","nonce":3296405141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6803","nonce":3296405141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6807","nonce":3296405141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6805","nonce":3296405141}]},"public_addr":"172.21.15.3:6801/3296405141","cluster_addr":"172.21.15.3:6803/3296405141","heartbeat_back_addr":"172.21.15.3:6807/3296405141","heartbeat_front_addr":"172.21.15.3:6805/3296405141","state":["exists","up"]},{"osd":1,"uuid":"89df5105-5900-4ffa-b8c1-6f480865a085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6801","nonce":1777872358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6803","nonce":1777872358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6810","nonce":1777872358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6805","nonce":1777872358}]},"public_addr":"172.21.15.203:6801/1777872358","cluster_addr":"172.21.15.203:6803/1777872358","heartbeat_back_addr":"172.21.15.203:6810/1777872358","heartbeat_front_addr":"172.21.15.203:6805/1777872358","state":["exists","up"]},{"osd":2,"uuid":"15f83334-8efc-40c2-af0c-eeed45923086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6817","nonce":1069390381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6819","nonce":1069390381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6823","nonce":1069390381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6821","nonce":1069390381}]},"public_addr":"172.21.15.203:6817/1069390381","cluster_addr":"172.21.15.203:6819/1069390381","heartbeat_back_addr":"172.21.15.203:6823/1069390381","heartbeat_front_addr":"172.21.15.203:6821/1069390381","state":["exists","up"]},{"osd":3,"uuid":"6a27ac8e-f957-4455-b477-3090081766c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6808","nonce":1072938612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6811","nonce":1072938612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6815","nonce":1072938612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6813","nonce":1072938612}]},"public_addr":"172.21.15.203:6808/1072938612","cluster_addr":"172.21.15.203:6811/1072938612","heartbeat_back_addr":"172.21.15.203:6815/1072938612","heartbeat_front_addr":"172.21.15.203:6813/1072938612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.224040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.344461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.319792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.332782+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.899+0000 7ff9e56c3640 10 mgrc shutdown 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.899+0000 7ff9e56c3640 1 -- 172.21.15.3:0/856339542 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff9cc0af260 msgr2=0x7ff9cc0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.899+0000 7ff9e56c3640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff9cc0af260 0x7ff9cc0b1720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0036000 tx=0x7ff9d0030000 comp rx=0 tx=0).stop 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.899+0000 7ff9e56c3640 1 -- 172.21.15.3:0/856339542 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9e0144cb0 msgr2=0x7ff9e0140450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.899+0000 7ff9e56c3640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9e0144cb0 0x7ff9e0140450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800c370 tx=0x7ff9d800c830 comp rx=0 tx=0).stop 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.900+0000 7ff9e56c3640 1 -- 172.21.15.3:0/856339542 shutdown_connections 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.900+0000 7ff9e56c3640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7ff9cc0af260 0x7ff9cc0b1720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.900+0000 7ff9e56c3640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff9e0144cb0 0x7ff9e0140450 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.900+0000 7ff9e56c3640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff9e00a7180 0x7ff9e00a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.900+0000 7ff9e56c3640 1 --2- 172.21.15.3:0/856339542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9e00a6d10 0x7ff9e00a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.900+0000 7ff9e56c3640 1 -- 172.21.15.3:0/856339542 >> 172.21.15.3:0/856339542 conn(0x7ff9e00a8c40 msgr2=0x7ff9e006fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:36.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.900+0000 7ff9e56c3640 1 -- 172.21.15.3:0/856339542 shutdown_connections 2024-05-21T17:41:36.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:36.900+0000 7ff9e56c3640 1 -- 172.21.15.3:0/856339542 wait complete. 2024-05-21T17:41:36.912 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-21T17:41:36.912 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T17:41:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.014+0000 7f89f4eed640 1 Processor -- start 2024-05-21T17:41:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.014+0000 7f89f4eed640 1 -- start start 2024-05-21T17:41:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89f4eed640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89f00ac330 0x7f89f00ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89f4eed640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 0x7f89f0149b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89f4eed640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f0146b20 con 0x7f89f014a040 2024-05-21T17:41:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89f4eed640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f89f00a8530 con 0x7f89f00ac330 2024-05-21T17:41:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89f4eed640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f89f0146de0 con 0x7f89f0149720 2024-05-21T17:41:37.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89ee59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89f00ac330 0x7f89f00ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89e7fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 0x7f89f0149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89ee59c640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f89f00ac330 0x7f89f00ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51372/0 (socket says 172.21.15.3:51372) 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89ee59c640 1 -- 172.21.15.3:0/2724793375 learned_addr learned my addr 172.21.15.3:0/2724793375 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89ee59c640 1 -- 172.21.15.3:0/2724793375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 msgr2=0x7f89f0149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89ee59c640 1 --2- 172.21.15.3:0/2724793375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 0x7f89f0149b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89ee59c640 1 -- 172.21.15.3:0/2724793375 >> v1:172.21.15.203:6789/0 conn(0x7f89f014a040 legacy=0x7f89f0145640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89ee59c640 1 -- 172.21.15.3:0/2724793375 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f014a4d0 con 0x7f89f00ac330 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.015+0000 7f89e7fff640 1 --2- 172.21.15.3:0/2724793375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 0x7f89f0149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.016+0000 7f89ee59c640 1 --2- 172.21.15.3:0/2724793375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 0x7f89f00ad480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009870 tx=0x7f89e002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1d04ed43e954ab server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.016+0000 7f89edd9b640 1 -- 172.21.15.3:0/2724793375 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0035020 con 0x7f89f00ac330 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.016+0000 7f89edd9b640 1 -- 172.21.15.3:0/2724793375 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e0031640 con 0x7f89f00ac330 2024-05-21T17:41:37.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.016+0000 7f89edd9b640 1 -- 172.21.15.3:0/2724793375 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e002d040 con 0x7f89f00ac330 2024-05-21T17:41:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.016+0000 7f89f4eed640 1 -- 172.21.15.3:0/2724793375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 msgr2=0x7f89f00ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.017+0000 7f89f4eed640 1 --2- 172.21.15.3:0/2724793375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 0x7f89f00ad480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009870 tx=0x7f89e002b8f0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.017+0000 7f89f4eed640 1 -- 172.21.15.3:0/2724793375 shutdown_connections 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.017+0000 7f89f4eed640 1 --2- 172.21.15.3:0/2724793375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 0x7f89f0149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.017+0000 7f89f4eed640 1 --2- 172.21.15.3:0/2724793375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 0x7f89f00ad480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.017+0000 7f89f4eed640 1 -- 172.21.15.3:0/2724793375 >> 172.21.15.3:0/2724793375 conn(0x7f89f00aab00 msgr2=0x7f89f00aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.017+0000 7f89f4eed640 1 -- 172.21.15.3:0/2724793375 shutdown_connections 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.017+0000 7f89f4eed640 1 -- 172.21.15.3:0/2724793375 wait complete. 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.018+0000 7f89f4eed640 1 Processor -- start 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.018+0000 7f89f4eed640 1 -- start start 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.018+0000 7f89f4eed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 0x7f89f01cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.018+0000 7f89f4eed640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 0x7f89f01d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.018+0000 7f89f4eed640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89f014a040 0x7f89f0138170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.018+0000 7f89f4eed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f89f006ccf0 con 0x7f89f00ac330 2024-05-21T17:41:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.018+0000 7f89f4eed640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f89f00ac830 con 0x7f89f014a040 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.018+0000 7f89f4eed640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f89f006ce70 con 0x7f89f0149720 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.018+0000 7f89ee59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 0x7f89f01cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89e7fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 0x7f89f01d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89ee59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 0x7f89f01cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51384/0 (socket says 172.21.15.3:51384) 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89ee59c640 1 -- 172.21.15.3:0/3742054128 learned_addr learned my addr 172.21.15.3:0/3742054128 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89eed9d640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89f014a040 0x7f89f0138170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89ee59c640 1 -- 172.21.15.3:0/3742054128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 msgr2=0x7f89f01d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89ee59c640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 0x7f89f01d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89ee59c640 1 -- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89f014a040 msgr2=0x7f89f0138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89ee59c640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89f014a040 0x7f89f0138170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89ee59c640 1 -- 172.21.15.3:0/3742054128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f013bd50 con 0x7f89f00ac330 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89e7fff640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 0x7f89f01d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89eed9d640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89f014a040 0x7f89f0138170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89ee59c640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 0x7f89f01cf7d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89e002b930 tx=0x7f89e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89e6ffd640 1 -- 172.21.15.3:0/3742054128 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0040070 con 0x7f89f00ac330 2024-05-21T17:41:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89e6ffd640 1 -- 172.21.15.3:0/3742054128 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e003e070 con 0x7f89f00ac330 2024-05-21T17:41:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89f4eed640 1 -- 172.21.15.3:0/3742054128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f013c010 con 0x7f89f00ac330 2024-05-21T17:41:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89e6ffd640 1 -- 172.21.15.3:0/3742054128 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0035040 con 0x7f89f00ac330 2024-05-21T17:41:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.019+0000 7f89f4eed640 1 -- 172.21.15.3:0/3742054128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f0139890 con 0x7f89f00ac330 2024-05-21T17:41:37.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.020+0000 7f89f4eed640 1 -- 172.21.15.3:0/3742054128 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f0069a90 con 0x7f89f00ac330 2024-05-21T17:41:37.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.022+0000 7f89e6ffd640 1 -- 172.21.15.3:0/3742054128 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f89e0038020 con 0x7f89f00ac330 2024-05-21T17:41:37.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.022+0000 7f89e6ffd640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:37.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.022+0000 7f89e6ffd640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:37.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.022+0000 7f89e6ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:37.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.022+0000 7f89e6ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:37.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.022+0000 7f89e6ffd640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f89d00af260 0x7f89d00b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.023+0000 7f89e6ffd640 1 -- 172.21.15.3:0/3742054128 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f89e00fe530 con 0x7f89f00ac330 2024-05-21T17:41:37.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.023+0000 7f89e7fff640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f89d00af260 0x7f89d00b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.023+0000 7f89e7fff640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f89d00af260 0x7f89d00b1720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f89f01475d0 tx=0x7f89d8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:37.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.026+0000 7f89e6ffd640 1 -- 172.21.15.3:0/3742054128 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f89e00c82e0 con 0x7f89f00ac330 2024-05-21T17:41:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.212+0000 7f89f4eed640 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2024-05-21T17:41:37.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.212+0000 7f89f4eed640 1 -- 172.21.15.3:0/3742054128 --> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f89f00a7780 con 0x7f89d00af260 2024-05-21T17:41:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.219+0000 7f89e6ffd640 1 -- 172.21.15.3:0/3742054128 <== mgr.4102 v2:172.21.15.203:6824/4157202067 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255676 (secure 0 0 0) 0x7f89f00a7780 con 0x7f89d00af260 2024-05-21T17:41:37.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.219+0000 7f89e6ffd640 20 mgrc handle_command_reply tid 0 r 0 2024-05-21T17:41:37.224 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:37.225 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-21T17:41:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 10 mgrc shutdown 2024-05-21T17:41:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 1 -- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f89d00af260 msgr2=0x7f89d00b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f89d00af260 0x7f89d00b1720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f89f01475d0 tx=0x7f89d8009190 comp rx=0 tx=0).stop 2024-05-21T17:41:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 1 -- 172.21.15.3:0/3742054128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 msgr2=0x7f89f01cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 0x7f89f01cf7d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89e002b930 tx=0x7f89e00046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 1 -- 172.21.15.3:0/3742054128 shutdown_connections 2024-05-21T17:41:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f89d00af260 0x7f89d00b1720 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89f014a040 0x7f89f0138170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f89f0149720 0x7f89f01d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 1 --2- 172.21.15.3:0/3742054128 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f89f00ac330 0x7f89f01cf7d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.222+0000 7f89f4eed640 1 -- 172.21.15.3:0/3742054128 >> 172.21.15.3:0/3742054128 conn(0x7f89f00aab00 msgr2=0x7f89f0145ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.223+0000 7f89f4eed640 1 -- 172.21.15.3:0/3742054128 shutdown_connections 2024-05-21T17:41:37.228 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.223+0000 7f89f4eed640 1 -- 172.21.15.3:0/3742054128 wait complete. 2024-05-21T17:41:37.235 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-05-21T17:41:35.518866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108656,"kb_used_data":528,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377378704,"statfs":{"total":386547056640,"available":386435792896,"internally_reserved":0,"allocated":540672,"data_stored":1304940,"data_compressed":17264,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":6377,"internal_metadata":110618391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.352968"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":102,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-21T17:41:28.155815+0000","last_change":"2024-05-21T17:41:27.156765+0000","last_active":"2024-05-21T17:41:28.155815+0000","last_peered":"2024-05-21T17:41:28.155815+0000","last_clean":"2024-05-21T17:41:28.155815+0000","last_became_active":"2024-05-21T17:41:27.156385+0000","last_became_peered":"2024-05-21T17:41:27.156385+0000","last_unstale":"2024-05-21T17:41:28.155815+0000","last_undegraded":"2024-05-21T17:41:28.155815+0000","last_fullsized":"2024-05-21T17:41:28.155815+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-21T17:41:28.203650+0000","last_change":"2024-05-21T17:41:27.156565+0000","last_active":"2024-05-21T17:41:28.203650+0000","last_peered":"2024-05-21T17:41:28.203650+0000","last_clean":"2024-05-21T17:41:28.203650+0000","last_became_active":"2024-05-21T17:41:27.156230+0000","last_became_peered":"2024-05-21T17:41:27.156230+0000","last_unstale":"2024-05-21T17:41:28.203650+0000","last_undegraded":"2024-05-21T17:41:28.203650+0000","last_fullsized":"2024-05-21T17:41:28.203650+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:27.800826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-21T17:41:28.203616+0000","last_change":"2024-05-21T17:41:27.155004+0000","last_active":"2024-05-21T17:41:28.203616+0000","last_peered":"2024-05-21T17:41:28.203616+0000","last_clean":"2024-05-21T17:41:28.203616+0000","last_became_active":"2024-05-21T17:41:27.154231+0000","last_became_peered":"2024-05-21T17:41:27.154231+0000","last_unstale":"2024-05-21T17:41:28.203616+0000","last_undegraded":"2024-05-21T17:41:28.203616+0000","last_fullsized":"2024-05-21T17:41:28.203616+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:16:06.123434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-21T17:41:28.575729+0000","last_change":"2024-05-21T17:41:27.159363+0000","last_active":"2024-05-21T17:41:28.575729+0000","last_peered":"2024-05-21T17:41:28.575729+0000","last_clean":"2024-05-21T17:41:28.575729+0000","last_became_active":"2024-05-21T17:41:27.159108+0000","last_became_peered":"2024-05-21T17:41:27.159108+0000","last_unstale":"2024-05-21T17:41:28.575729+0000","last_undegraded":"2024-05-21T17:41:28.575729+0000","last_fullsized":"2024-05-21T17:41:28.575729+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-21T17:41:28.203637+0000","last_change":"2024-05-21T17:41:27.154942+0000","last_active":"2024-05-21T17:41:28.203637+0000","last_peered":"2024-05-21T17:41:28.203637+0000","last_clean":"2024-05-21T17:41:28.203637+0000","last_became_active":"2024-05-21T17:41:27.154055+0000","last_became_peered":"2024-05-21T17:41:27.154055+0000","last_unstale":"2024-05-21T17:41:28.203637+0000","last_undegraded":"2024-05-21T17:41:28.203637+0000","last_fullsized":"2024-05-21T17:41:28.203637+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:58:33.630777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-21T17:41:28.157061+0000","last_change":"2024-05-21T17:41:27.154907+0000","last_active":"2024-05-21T17:41:28.157061+0000","last_peered":"2024-05-21T17:41:28.157061+0000","last_clean":"2024-05-21T17:41:28.157061+0000","last_became_active":"2024-05-21T17:41:27.154659+0000","last_became_peered":"2024-05-21T17:41:27.154659+0000","last_unstale":"2024-05-21T17:41:28.157061+0000","last_undegraded":"2024-05-21T17:41:28.157061+0000","last_fullsized":"2024-05-21T17:41:28.157061+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:28:47.634110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-21T17:41:28.203684+0000","last_change":"2024-05-21T17:41:27.154725+0000","last_active":"2024-05-21T17:41:28.203684+0000","last_peered":"2024-05-21T17:41:28.203684+0000","last_clean":"2024-05-21T17:41:28.203684+0000","last_became_active":"2024-05-21T17:41:27.154339+0000","last_became_peered":"2024-05-21T17:41:27.154339+0000","last_unstale":"2024-05-21T17:41:28.203684+0000","last_undegraded":"2024-05-21T17:41:28.203684+0000","last_fullsized":"2024-05-21T17:41:28.203684+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:47:27.935607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-21T17:41:28.157161+0000","last_change":"2024-05-21T17:41:27.156568+0000","last_active":"2024-05-21T17:41:28.157161+0000","last_peered":"2024-05-21T17:41:28.157161+0000","last_clean":"2024-05-21T17:41:28.157161+0000","last_became_active":"2024-05-21T17:41:27.156293+0000","last_became_peered":"2024-05-21T17:41:27.156293+0000","last_unstale":"2024-05-21T17:41:28.157161+0000","last_undegraded":"2024-05-21T17:41:28.157161+0000","last_fullsized":"2024-05-21T17:41:28.157161+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-21T17:41:28.203652+0000","last_change":"2024-05-21T17:41:27.156830+0000","last_active":"2024-05-21T17:41:28.203652+0000","last_peered":"2024-05-21T17:41:28.203652+0000","last_clean":"2024-05-21T17:41:28.203652+0000","last_became_active":"2024-05-21T17:41:27.156384+0000","last_became_peered":"2024-05-21T17:41:27.156384+0000","last_unstale":"2024-05-21T17:41:28.203652+0000","last_undegraded":"2024-05-21T17:41:28.203652+0000","last_fullsized":"2024-05-21T17:41:28.203652+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:21:15.563350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:32.164627+0000","last_change":"2024-05-21T17:41:32.164627+0000","last_active":"2024-05-21T17:41:32.164627+0000","last_peered":"2024-05-21T17:41:32.164627+0000","last_clean":"2024-05-21T17:41:32.164627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:32.164627+0000","last_undegraded":"2024-05-21T17:41:32.164627+0000","last_fullsized":"2024-05-21T17:41:32.164627+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-21T17:41:31.159346+0000","last_change":"2024-05-21T17:41:31.159346+0000","last_active":"2024-05-21T17:41:31.159346+0000","last_peered":"2024-05-21T17:41:31.159346+0000","last_clean":"2024-05-21T17:41:31.159346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-21T17:41:31.159346+0000","last_undegraded":"2024-05-21T17:41:31.159346+0000","last_fullsized":"2024-05-21T17:41:31.159346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17264,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":106496,"data_stored":31051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":163840,"data_stored":621419,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":163840,"data_stored":621419,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":106496,"data_stored":31051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T17:41:37.239 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T17:41:37.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.377+0000 7f0cfd99a640 1 Processor -- start 2024-05-21T17:41:37.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.377+0000 7f0cfd99a640 1 -- start start 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cfd99a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf80a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cfd99a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0cf80a53f0 0x7f0cf80adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cfd99a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf80a8fe0 con 0x7f0cf80a7b10 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cfd99a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf80a92a0 con 0x7f0cf80a53f0 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cfd99a640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf806ed60 con 0x7f0cf80ac570 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf6ffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf80a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf67fc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f0cf80a53f0 0x7f0cf80adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf6ffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf80a8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40832/0 (socket says 172.21.15.3:40832) 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf6ffd640 1 -- 172.21.15.3:0/2301083724 learned_addr learned my addr 172.21.15.3:0/2301083724 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf77fe640 1 --1- 172.21.15.3:0/2301083724 >> v1:172.21.15.3:6789/0 conn(0x7f0cf80a7b10 0x7f0cf80a7f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60666/0 (socket says 172.21.15.3:60666) 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf6ffd640 1 -- 172.21.15.3:0/2301083724 >> v2:172.21.15.3:3300/0 conn(0x7f0cf80a53f0 msgr2=0x7f0cf80adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:37.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf6ffd640 1 --2- 172.21.15.3:0/2301083724 >> v2:172.21.15.3:3300/0 conn(0x7f0cf80a53f0 0x7f0cf80adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf6ffd640 1 -- 172.21.15.3:0/2301083724 >> v1:172.21.15.3:6789/0 conn(0x7f0cf80a7b10 legacy=0x7f0cf80a7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf6ffd640 1 -- 172.21.15.3:0/2301083724 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf80ac950 con 0x7f0cf80ac570 2024-05-21T17:41:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf5ffb640 1 -- 172.21.15.3:0/2301083724 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2775474581 0 0) 0x7f0cf80a8fe0 con 0x7f0cf80a7b10 2024-05-21T17:41:37.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.378+0000 7f0cf67fc640 1 --2- 172.21.15.3:0/2301083724 >> v2:172.21.15.3:3300/0 conn(0x7f0cf80a53f0 0x7f0cf80adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.379+0000 7f0cf6ffd640 1 --2- 172.21.15.3:0/2301083724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf80a8990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009870 tx=0x7f0ce802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a6d3080c387b4f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.379+0000 7f0cf5ffb640 1 -- 172.21.15.3:0/2301083724 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8035020 con 0x7f0cf80ac570 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.379+0000 7f0cf5ffb640 1 -- 172.21.15.3:0/2301083724 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce8031640 con 0x7f0cf80ac570 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.379+0000 7f0cf5ffb640 1 -- 172.21.15.3:0/2301083724 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce802d040 con 0x7f0cf80ac570 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.380+0000 7f0cfd99a640 1 -- 172.21.15.3:0/2301083724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 msgr2=0x7f0cf80a8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.380+0000 7f0cfd99a640 1 --2- 172.21.15.3:0/2301083724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf80a8990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009870 tx=0x7f0ce802b8f0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.380+0000 7f0cfd99a640 1 -- 172.21.15.3:0/2301083724 shutdown_connections 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.380+0000 7f0cfd99a640 1 --2- 172.21.15.3:0/2301083724 >> v2:172.21.15.3:3300/0 conn(0x7f0cf80a53f0 0x7f0cf80adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.380+0000 7f0cfd99a640 1 --2- 172.21.15.3:0/2301083724 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf80a8990 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.380+0000 7f0cfd99a640 1 -- 172.21.15.3:0/2301083724 >> 172.21.15.3:0/2301083724 conn(0x7f0cf80bb9e0 msgr2=0x7f0cf80bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:37.384 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.380+0000 7f0cfd99a640 1 -- 172.21.15.3:0/2301083724 shutdown_connections 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.380+0000 7f0cfd99a640 1 -- 172.21.15.3:0/2301083724 wait complete. 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.381+0000 7f0cfd99a640 1 Processor -- start 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.381+0000 7f0cfd99a640 1 -- start start 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cfd99a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf80a53f0 0x7f0cf814cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cfd99a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf80a7b10 0x7f0cf814a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cfd99a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf814adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cfd99a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf80ac180 con 0x7f0cf80a53f0 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cfd99a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf80ac000 con 0x7f0cf80a7b10 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cfd99a640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf8150bd0 con 0x7f0cf80ac570 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf80a53f0 0x7f0cf814cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf77fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf814adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf80a53f0 0x7f0cf814cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51408/0 (socket says 172.21.15.3:51408) 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf6ffd640 1 -- 172.21.15.3:0/819328707 learned_addr learned my addr 172.21.15.3:0/819328707 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:37.385 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf67fc640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf80a7b10 0x7f0cf814a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf6ffd640 1 -- 172.21.15.3:0/819328707 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 msgr2=0x7f0cf814adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf6ffd640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf814adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf6ffd640 1 -- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf80a7b10 msgr2=0x7f0cf814a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf6ffd640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf80a7b10 0x7f0cf814a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf6ffd640 1 -- 172.21.15.3:0/819328707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf8145410 con 0x7f0cf80a53f0 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf67fc640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf80a7b10 0x7f0cf814a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf77fe640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf814adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cf6ffd640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf80a53f0 0x7f0cf814cb60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80090b0 tx=0x7f0ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cdffff640 1 -- 172.21.15.3:0/819328707 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8040070 con 0x7f0cf80a53f0 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cdffff640 1 -- 172.21.15.3:0/819328707 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce80399f0 con 0x7f0cf80a53f0 2024-05-21T17:41:37.386 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cfd99a640 1 -- 172.21.15.3:0/819328707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf81468c0 con 0x7f0cf80a53f0 2024-05-21T17:41:37.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cdffff640 1 -- 172.21.15.3:0/819328707 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8035040 con 0x7f0cf80a53f0 2024-05-21T17:41:37.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.382+0000 7f0cfd99a640 1 -- 172.21.15.3:0/819328707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf814c660 con 0x7f0cf80a53f0 2024-05-21T17:41:37.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.383+0000 7f0cfd99a640 1 -- 172.21.15.3:0/819328707 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf8069a90 con 0x7f0cf80a53f0 2024-05-21T17:41:37.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.385+0000 7f0cdffff640 1 -- 172.21.15.3:0/819328707 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f0ce8038020 con 0x7f0cf80a53f0 2024-05-21T17:41:37.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.385+0000 7f0cdffff640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:37.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.386+0000 7f0cdffff640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:37.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.386+0000 7f0cdffff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:37.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.386+0000 7f0cdffff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:37.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.386+0000 7f0cdffff640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f0ce40af290 0x7f0ce40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.386+0000 7f0cdffff640 1 -- 172.21.15.3:0/819328707 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f0ce80fe740 con 0x7f0cf80a53f0 2024-05-21T17:41:37.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.386+0000 7f0cf67fc640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f0ce40af290 0x7f0ce40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.387+0000 7f0cf67fc640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f0ce40af290 0x7f0ce40b1750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80b95d0 tx=0x7f0cec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:37.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.389+0000 7f0cdffff640 1 -- 172.21.15.3:0/819328707 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0ce80c84f0 con 0x7f0cf80a53f0 2024-05-21T17:41:37.582 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.581+0000 7f0cfd99a640 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2024-05-21T17:41:37.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.581+0000 7f0cfd99a640 1 -- 172.21.15.3:0/819328707 --> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0cf80a78f0 con 0x7f0ce40af290 2024-05-21T17:41:37.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.588+0000 7f0cdffff640 1 -- 172.21.15.3:0/819328707 <== mgr.4102 v2:172.21.15.203:6824/4157202067 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+268665 (secure 0 0 0) 0x7f0cf80a78f0 con 0x7f0ce40af290 2024-05-21T17:41:37.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.588+0000 7f0cdffff640 20 mgrc handle_command_reply tid 0 r 0 2024-05-21T17:41:37.593 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:37.594 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-21T17:41:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 10 mgrc shutdown 2024-05-21T17:41:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 -- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f0ce40af290 msgr2=0x7f0ce40b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f0ce40af290 0x7f0ce40b1750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80b95d0 tx=0x7f0cec008040 comp rx=0 tx=0).stop 2024-05-21T17:41:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 -- 172.21.15.3:0/819328707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf80a53f0 msgr2=0x7f0cf814cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf80a53f0 0x7f0cf814cb60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80090b0 tx=0x7f0ce80046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 -- 172.21.15.3:0/819328707 shutdown_connections 2024-05-21T17:41:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f0ce40af290 0x7f0ce40b1750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f0cf80ac570 0x7f0cf814adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf80a7b10 0x7f0cf814a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 --2- 172.21.15.3:0/819328707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0cf80a53f0 0x7f0cf814cb60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 -- 172.21.15.3:0/819328707 >> 172.21.15.3:0/819328707 conn(0x7f0cf80bb9e0 msgr2=0x7f0cf8142be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:37.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 -- 172.21.15.3:0/819328707 shutdown_connections 2024-05-21T17:41:37.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.592+0000 7f0cfd99a640 1 -- 172.21.15.3:0/819328707 wait complete. 2024-05-21T17:41:37.604 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-05-21T17:41:37.519209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108852,"kb_used_data":756,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377378508,"statfs":{"total":386547056640,"available":386435592192,"internally_reserved":0,"allocated":774144,"data_stored":1383247,"data_compressed":17264,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":6377,"internal_metadata":110618391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001212"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181633+0000","last_change":"2024-05-21T17:41:33.197173+0000","last_active":"2024-05-21T17:41:35.181633+0000","last_peered":"2024-05-21T17:41:35.181633+0000","last_clean":"2024-05-21T17:41:35.181633+0000","last_became_active":"2024-05-21T17:41:33.196781+0000","last_became_peered":"2024-05-21T17:41:33.196781+0000","last_unstale":"2024-05-21T17:41:35.181633+0000","last_undegraded":"2024-05-21T17:41:35.181633+0000","last_fullsized":"2024-05-21T17:41:35.181633+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:12:57.454391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181522+0000","last_change":"2024-05-21T17:41:33.197965+0000","last_active":"2024-05-21T17:41:35.181522+0000","last_peered":"2024-05-21T17:41:35.181522+0000","last_clean":"2024-05-21T17:41:35.181522+0000","last_became_active":"2024-05-21T17:41:33.197693+0000","last_became_peered":"2024-05-21T17:41:33.197693+0000","last_unstale":"2024-05-21T17:41:35.181522+0000","last_undegraded":"2024-05-21T17:41:35.181522+0000","last_fullsized":"2024-05-21T17:41:35.181522+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:40:32.331027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504514+0000","last_change":"2024-05-21T17:41:33.212529+0000","last_active":"2024-05-21T17:41:35.504514+0000","last_peered":"2024-05-21T17:41:35.504514+0000","last_clean":"2024-05-21T17:41:35.504514+0000","last_became_active":"2024-05-21T17:41:33.211026+0000","last_became_peered":"2024-05-21T17:41:33.211026+0000","last_unstale":"2024-05-21T17:41:35.504514+0000","last_undegraded":"2024-05-21T17:41:35.504514+0000","last_fullsized":"2024-05-21T17:41:35.504514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:15:01.504097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196450+0000","last_change":"2024-05-21T17:41:33.191234+0000","last_active":"2024-05-21T17:41:34.196450+0000","last_peered":"2024-05-21T17:41:34.196450+0000","last_clean":"2024-05-21T17:41:34.196450+0000","last_became_active":"2024-05-21T17:41:33.190138+0000","last_became_peered":"2024-05-21T17:41:33.190138+0000","last_unstale":"2024-05-21T17:41:34.196450+0000","last_undegraded":"2024-05-21T17:41:34.196450+0000","last_fullsized":"2024-05-21T17:41:34.196450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:31.958772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504393+0000","last_change":"2024-05-21T17:41:33.203349+0000","last_active":"2024-05-21T17:41:35.504393+0000","last_peered":"2024-05-21T17:41:35.504393+0000","last_clean":"2024-05-21T17:41:35.504393+0000","last_became_active":"2024-05-21T17:41:33.203059+0000","last_became_peered":"2024-05-21T17:41:33.203059+0000","last_unstale":"2024-05-21T17:41:35.504393+0000","last_undegraded":"2024-05-21T17:41:35.504393+0000","last_fullsized":"2024-05-21T17:41:35.504393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:10:26.347216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504144+0000","last_change":"2024-05-21T17:41:33.208230+0000","last_active":"2024-05-21T17:41:35.504144+0000","last_peered":"2024-05-21T17:41:35.504144+0000","last_clean":"2024-05-21T17:41:35.504144+0000","last_became_active":"2024-05-21T17:41:33.207955+0000","last_became_peered":"2024-05-21T17:41:33.207955+0000","last_unstale":"2024-05-21T17:41:35.504144+0000","last_undegraded":"2024-05-21T17:41:35.504144+0000","last_fullsized":"2024-05-21T17:41:35.504144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:42:25.073971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503817+0000","last_change":"2024-05-21T17:41:33.208443+0000","last_active":"2024-05-21T17:41:35.503817+0000","last_peered":"2024-05-21T17:41:35.503817+0000","last_clean":"2024-05-21T17:41:35.503817+0000","last_became_active":"2024-05-21T17:41:33.208038+0000","last_became_peered":"2024-05-21T17:41:33.208038+0000","last_unstale":"2024-05-21T17:41:35.503817+0000","last_undegraded":"2024-05-21T17:41:35.503817+0000","last_fullsized":"2024-05-21T17:41:35.503817+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:14:16.199161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195697+0000","last_change":"2024-05-21T17:41:33.202443+0000","last_active":"2024-05-21T17:41:34.195697+0000","last_peered":"2024-05-21T17:41:34.195697+0000","last_clean":"2024-05-21T17:41:34.195697+0000","last_became_active":"2024-05-21T17:41:33.200436+0000","last_became_peered":"2024-05-21T17:41:33.200436+0000","last_unstale":"2024-05-21T17:41:34.195697+0000","last_undegraded":"2024-05-21T17:41:34.195697+0000","last_fullsized":"2024-05-21T17:41:34.195697+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:41:08.380018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197981+0000","last_change":"2024-05-21T17:41:33.185540+0000","last_active":"2024-05-21T17:41:34.197981+0000","last_peered":"2024-05-21T17:41:34.197981+0000","last_clean":"2024-05-21T17:41:34.197981+0000","last_became_active":"2024-05-21T17:41:33.185269+0000","last_became_peered":"2024-05-21T17:41:33.185269+0000","last_unstale":"2024-05-21T17:41:34.197981+0000","last_undegraded":"2024-05-21T17:41:34.197981+0000","last_fullsized":"2024-05-21T17:41:34.197981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:29:07.891173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196019+0000","last_change":"2024-05-21T17:41:33.204222+0000","last_active":"2024-05-21T17:41:34.196019+0000","last_peered":"2024-05-21T17:41:34.196019+0000","last_clean":"2024-05-21T17:41:34.196019+0000","last_became_active":"2024-05-21T17:41:33.203828+0000","last_became_peered":"2024-05-21T17:41:33.203828+0000","last_unstale":"2024-05-21T17:41:34.196019+0000","last_undegraded":"2024-05-21T17:41:34.196019+0000","last_fullsized":"2024-05-21T17:41:34.196019+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:09.846517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196360+0000","last_change":"2024-05-21T17:41:33.190185+0000","last_active":"2024-05-21T17:41:34.196360+0000","last_peered":"2024-05-21T17:41:34.196360+0000","last_clean":"2024-05-21T17:41:34.196360+0000","last_became_active":"2024-05-21T17:41:33.189865+0000","last_became_peered":"2024-05-21T17:41:33.189865+0000","last_unstale":"2024-05-21T17:41:34.196360+0000","last_undegraded":"2024-05-21T17:41:34.196360+0000","last_fullsized":"2024-05-21T17:41:34.196360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:36:50.231113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181658+0000","last_change":"2024-05-21T17:41:33.212714+0000","last_active":"2024-05-21T17:41:35.181658+0000","last_peered":"2024-05-21T17:41:35.181658+0000","last_clean":"2024-05-21T17:41:35.181658+0000","last_became_active":"2024-05-21T17:41:33.212447+0000","last_became_peered":"2024-05-21T17:41:33.212447+0000","last_unstale":"2024-05-21T17:41:35.181658+0000","last_undegraded":"2024-05-21T17:41:35.181658+0000","last_fullsized":"2024-05-21T17:41:35.181658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:06.343413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197751+0000","last_change":"2024-05-21T17:41:33.194642+0000","last_active":"2024-05-21T17:41:34.197751+0000","last_peered":"2024-05-21T17:41:34.197751+0000","last_clean":"2024-05-21T17:41:34.197751+0000","last_became_active":"2024-05-21T17:41:33.194331+0000","last_became_peered":"2024-05-21T17:41:33.194331+0000","last_unstale":"2024-05-21T17:41:34.197751+0000","last_undegraded":"2024-05-21T17:41:34.197751+0000","last_fullsized":"2024-05-21T17:41:34.197751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:54:27.905968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195573+0000","last_change":"2024-05-21T17:41:33.200546+0000","last_active":"2024-05-21T17:41:34.195573+0000","last_peered":"2024-05-21T17:41:34.195573+0000","last_clean":"2024-05-21T17:41:34.195573+0000","last_became_active":"2024-05-21T17:41:33.199969+0000","last_became_peered":"2024-05-21T17:41:33.199969+0000","last_unstale":"2024-05-21T17:41:34.195573+0000","last_undegraded":"2024-05-21T17:41:34.195573+0000","last_fullsized":"2024-05-21T17:41:34.195573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:06.343413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195553+0000","last_change":"2024-05-21T17:41:33.200283+0000","last_active":"2024-05-21T17:41:34.195553+0000","last_peered":"2024-05-21T17:41:34.195553+0000","last_clean":"2024-05-21T17:41:34.195553+0000","last_became_active":"2024-05-21T17:41:33.199931+0000","last_became_peered":"2024-05-21T17:41:33.199931+0000","last_unstale":"2024-05-21T17:41:34.195553+0000","last_undegraded":"2024-05-21T17:41:34.195553+0000","last_fullsized":"2024-05-21T17:41:34.195553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:12:57.454391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503627+0000","last_change":"2024-05-21T17:41:33.208399+0000","last_active":"2024-05-21T17:41:35.503627+0000","last_peered":"2024-05-21T17:41:35.503627+0000","last_clean":"2024-05-21T17:41:35.503627+0000","last_became_active":"2024-05-21T17:41:33.208104+0000","last_became_peered":"2024-05-21T17:41:33.208104+0000","last_unstale":"2024-05-21T17:41:35.503627+0000","last_undegraded":"2024-05-21T17:41:35.503627+0000","last_fullsized":"2024-05-21T17:41:35.503627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:14:30.272478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196463+0000","last_change":"2024-05-21T17:41:33.200479+0000","last_active":"2024-05-21T17:41:34.196463+0000","last_peered":"2024-05-21T17:41:34.196463+0000","last_clean":"2024-05-21T17:41:34.196463+0000","last_became_active":"2024-05-21T17:41:33.199926+0000","last_became_peered":"2024-05-21T17:41:33.199926+0000","last_unstale":"2024-05-21T17:41:34.196463+0000","last_undegraded":"2024-05-21T17:41:34.196463+0000","last_fullsized":"2024-05-21T17:41:34.196463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:03:55.087212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504684+0000","last_change":"2024-05-21T17:41:33.208923+0000","last_active":"2024-05-21T17:41:35.504684+0000","last_peered":"2024-05-21T17:41:35.504684+0000","last_clean":"2024-05-21T17:41:35.504684+0000","last_became_active":"2024-05-21T17:41:33.208377+0000","last_became_peered":"2024-05-21T17:41:33.208377+0000","last_unstale":"2024-05-21T17:41:35.504684+0000","last_undegraded":"2024-05-21T17:41:35.504684+0000","last_fullsized":"2024-05-21T17:41:35.504684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:49.839600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197601+0000","last_change":"2024-05-21T17:41:33.214811+0000","last_active":"2024-05-21T17:41:34.197601+0000","last_peered":"2024-05-21T17:41:34.197601+0000","last_clean":"2024-05-21T17:41:34.197601+0000","last_became_active":"2024-05-21T17:41:33.214502+0000","last_became_peered":"2024-05-21T17:41:33.214502+0000","last_unstale":"2024-05-21T17:41:34.197601+0000","last_undegraded":"2024-05-21T17:41:34.197601+0000","last_fullsized":"2024-05-21T17:41:34.197601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:11:24.435803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196070+0000","last_change":"2024-05-21T17:41:33.190432+0000","last_active":"2024-05-21T17:41:34.196070+0000","last_peered":"2024-05-21T17:41:34.196070+0000","last_clean":"2024-05-21T17:41:34.196070+0000","last_became_active":"2024-05-21T17:41:33.189727+0000","last_became_peered":"2024-05-21T17:41:33.189727+0000","last_unstale":"2024-05-21T17:41:34.196070+0000","last_undegraded":"2024-05-21T17:41:34.196070+0000","last_fullsized":"2024-05-21T17:41:34.196070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:45:43.258694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195941+0000","last_change":"2024-05-21T17:41:33.206591+0000","last_active":"2024-05-21T17:41:34.195941+0000","last_peered":"2024-05-21T17:41:34.195941+0000","last_clean":"2024-05-21T17:41:34.195941+0000","last_became_active":"2024-05-21T17:41:33.205156+0000","last_became_peered":"2024-05-21T17:41:33.205156+0000","last_unstale":"2024-05-21T17:41:34.195941+0000","last_undegraded":"2024-05-21T17:41:34.195941+0000","last_fullsized":"2024-05-21T17:41:34.195941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:55:26.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195772+0000","last_change":"2024-05-21T17:41:33.200833+0000","last_active":"2024-05-21T17:41:34.195772+0000","last_peered":"2024-05-21T17:41:34.195772+0000","last_clean":"2024-05-21T17:41:34.195772+0000","last_became_active":"2024-05-21T17:41:33.200208+0000","last_became_peered":"2024-05-21T17:41:33.200208+0000","last_unstale":"2024-05-21T17:41:34.195772+0000","last_undegraded":"2024-05-21T17:41:34.195772+0000","last_fullsized":"2024-05-21T17:41:34.195772+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:28:20.455810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503916+0000","last_change":"2024-05-21T17:41:33.214524+0000","last_active":"2024-05-21T17:41:35.503916+0000","last_peered":"2024-05-21T17:41:35.503916+0000","last_clean":"2024-05-21T17:41:35.503916+0000","last_became_active":"2024-05-21T17:41:33.211351+0000","last_became_peered":"2024-05-21T17:41:33.211351+0000","last_unstale":"2024-05-21T17:41:35.503916+0000","last_undegraded":"2024-05-21T17:41:35.503916+0000","last_fullsized":"2024-05-21T17:41:35.503916+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:25:58.491664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195655+0000","last_change":"2024-05-21T17:41:33.194325+0000","last_active":"2024-05-21T17:41:34.195655+0000","last_peered":"2024-05-21T17:41:34.195655+0000","last_clean":"2024-05-21T17:41:34.195655+0000","last_became_active":"2024-05-21T17:41:33.193802+0000","last_became_peered":"2024-05-21T17:41:33.193802+0000","last_unstale":"2024-05-21T17:41:34.195655+0000","last_undegraded":"2024-05-21T17:41:34.195655+0000","last_fullsized":"2024-05-21T17:41:34.195655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:38:18.545593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196458+0000","last_change":"2024-05-21T17:41:33.209308+0000","last_active":"2024-05-21T17:41:34.196458+0000","last_peered":"2024-05-21T17:41:34.196458+0000","last_clean":"2024-05-21T17:41:34.196458+0000","last_became_active":"2024-05-21T17:41:33.209014+0000","last_became_peered":"2024-05-21T17:41:33.209014+0000","last_unstale":"2024-05-21T17:41:34.196458+0000","last_undegraded":"2024-05-21T17:41:34.196458+0000","last_fullsized":"2024-05-21T17:41:34.196458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:14:30.272478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197581+0000","last_change":"2024-05-21T17:41:33.197441+0000","last_active":"2024-05-21T17:41:34.197581+0000","last_peered":"2024-05-21T17:41:34.197581+0000","last_clean":"2024-05-21T17:41:34.197581+0000","last_became_active":"2024-05-21T17:41:33.196890+0000","last_became_peered":"2024-05-21T17:41:33.196890+0000","last_unstale":"2024-05-21T17:41:34.197581+0000","last_undegraded":"2024-05-21T17:41:34.197581+0000","last_fullsized":"2024-05-21T17:41:34.197581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:29:26.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505181+0000","last_change":"2024-05-21T17:41:33.214392+0000","last_active":"2024-05-21T17:41:35.505181+0000","last_peered":"2024-05-21T17:41:35.505181+0000","last_clean":"2024-05-21T17:41:35.505181+0000","last_became_active":"2024-05-21T17:41:33.213788+0000","last_became_peered":"2024-05-21T17:41:33.213788+0000","last_unstale":"2024-05-21T17:41:35.505181+0000","last_undegraded":"2024-05-21T17:41:35.505181+0000","last_fullsized":"2024-05-21T17:41:35.505181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:33:54.977766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198327+0000","last_change":"2024-05-21T17:41:33.197602+0000","last_active":"2024-05-21T17:41:34.198327+0000","last_peered":"2024-05-21T17:41:34.198327+0000","last_clean":"2024-05-21T17:41:34.198327+0000","last_became_active":"2024-05-21T17:41:33.196498+0000","last_became_peered":"2024-05-21T17:41:33.196498+0000","last_unstale":"2024-05-21T17:41:34.198327+0000","last_undegraded":"2024-05-21T17:41:34.198327+0000","last_fullsized":"2024-05-21T17:41:34.198327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:06.343413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197904+0000","last_change":"2024-05-21T17:41:33.194996+0000","last_active":"2024-05-21T17:41:34.197904+0000","last_peered":"2024-05-21T17:41:34.197904+0000","last_clean":"2024-05-21T17:41:34.197904+0000","last_became_active":"2024-05-21T17:41:33.194676+0000","last_became_peered":"2024-05-21T17:41:33.194676+0000","last_unstale":"2024-05-21T17:41:34.197904+0000","last_undegraded":"2024-05-21T17:41:34.197904+0000","last_fullsized":"2024-05-21T17:41:34.197904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:48.164464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196316+0000","last_change":"2024-05-21T17:41:33.209301+0000","last_active":"2024-05-21T17:41:34.196316+0000","last_peered":"2024-05-21T17:41:34.196316+0000","last_clean":"2024-05-21T17:41:34.196316+0000","last_became_active":"2024-05-21T17:41:33.209030+0000","last_became_peered":"2024-05-21T17:41:33.209030+0000","last_unstale":"2024-05-21T17:41:34.196316+0000","last_undegraded":"2024-05-21T17:41:34.196316+0000","last_fullsized":"2024-05-21T17:41:34.196316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:46:10.096945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197268+0000","last_change":"2024-05-21T17:41:33.185555+0000","last_active":"2024-05-21T17:41:34.197268+0000","last_peered":"2024-05-21T17:41:34.197268+0000","last_clean":"2024-05-21T17:41:34.197268+0000","last_became_active":"2024-05-21T17:41:33.185206+0000","last_became_peered":"2024-05-21T17:41:33.185206+0000","last_unstale":"2024-05-21T17:41:34.197268+0000","last_undegraded":"2024-05-21T17:41:34.197268+0000","last_fullsized":"2024-05-21T17:41:34.197268+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:53:00.383833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196388+0000","last_change":"2024-05-21T17:41:33.194078+0000","last_active":"2024-05-21T17:41:34.196388+0000","last_peered":"2024-05-21T17:41:34.196388+0000","last_clean":"2024-05-21T17:41:34.196388+0000","last_became_active":"2024-05-21T17:41:33.190990+0000","last_became_peered":"2024-05-21T17:41:33.190990+0000","last_unstale":"2024-05-21T17:41:34.196388+0000","last_undegraded":"2024-05-21T17:41:34.196388+0000","last_fullsized":"2024-05-21T17:41:34.196388+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:23:29.692427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504416+0000","last_change":"2024-05-21T17:41:33.203400+0000","last_active":"2024-05-21T17:41:35.504416+0000","last_peered":"2024-05-21T17:41:35.504416+0000","last_clean":"2024-05-21T17:41:35.504416+0000","last_became_active":"2024-05-21T17:41:33.203117+0000","last_became_peered":"2024-05-21T17:41:33.203117+0000","last_unstale":"2024-05-21T17:41:35.504416+0000","last_undegraded":"2024-05-21T17:41:35.504416+0000","last_fullsized":"2024-05-21T17:41:35.504416+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:30:00.800497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504541+0000","last_change":"2024-05-21T17:41:32.194821+0000","last_active":"2024-05-21T17:41:35.504541+0000","last_peered":"2024-05-21T17:41:35.504541+0000","last_clean":"2024-05-21T17:41:35.504541+0000","last_became_active":"2024-05-21T17:41:32.194501+0000","last_became_peered":"2024-05-21T17:41:32.194501+0000","last_unstale":"2024-05-21T17:41:35.504541+0000","last_undegraded":"2024-05-21T17:41:35.504541+0000","last_fullsized":"2024-05-21T17:41:35.504541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:32:31.739397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505001+0000","last_change":"2024-05-21T17:41:33.206606+0000","last_active":"2024-05-21T17:41:35.505001+0000","last_peered":"2024-05-21T17:41:35.505001+0000","last_clean":"2024-05-21T17:41:35.505001+0000","last_became_active":"2024-05-21T17:41:33.206194+0000","last_became_peered":"2024-05-21T17:41:33.206194+0000","last_unstale":"2024-05-21T17:41:35.505001+0000","last_undegraded":"2024-05-21T17:41:35.505001+0000","last_fullsized":"2024-05-21T17:41:35.505001+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:04:01.510328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181288+0000","last_change":"2024-05-21T17:41:32.183395+0000","last_active":"2024-05-21T17:41:35.181288+0000","last_peered":"2024-05-21T17:41:35.181288+0000","last_clean":"2024-05-21T17:41:35.181288+0000","last_became_active":"2024-05-21T17:41:32.183108+0000","last_became_peered":"2024-05-21T17:41:32.183108+0000","last_unstale":"2024-05-21T17:41:35.181288+0000","last_undegraded":"2024-05-21T17:41:35.181288+0000","last_fullsized":"2024-05-21T17:41:35.181288+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:33.195952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504687+0000","last_change":"2024-05-21T17:41:33.198913+0000","last_active":"2024-05-21T17:41:35.504687+0000","last_peered":"2024-05-21T17:41:35.504687+0000","last_clean":"2024-05-21T17:41:35.504687+0000","last_became_active":"2024-05-21T17:41:33.198607+0000","last_became_peered":"2024-05-21T17:41:33.198607+0000","last_unstale":"2024-05-21T17:41:35.504687+0000","last_undegraded":"2024-05-21T17:41:35.504687+0000","last_fullsized":"2024-05-21T17:41:35.504687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:14:01.670496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504299+0000","last_change":"2024-05-21T17:41:32.194461+0000","last_active":"2024-05-21T17:41:35.504299+0000","last_peered":"2024-05-21T17:41:35.504299+0000","last_clean":"2024-05-21T17:41:35.504299+0000","last_became_active":"2024-05-21T17:41:32.194194+0000","last_became_peered":"2024-05-21T17:41:32.194194+0000","last_unstale":"2024-05-21T17:41:35.504299+0000","last_undegraded":"2024-05-21T17:41:35.504299+0000","last_fullsized":"2024-05-21T17:41:35.504299+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:00:15.399735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198049+0000","last_change":"2024-05-21T17:41:33.198965+0000","last_active":"2024-05-21T17:41:34.198049+0000","last_peered":"2024-05-21T17:41:34.198049+0000","last_clean":"2024-05-21T17:41:34.198049+0000","last_became_active":"2024-05-21T17:41:33.197958+0000","last_became_peered":"2024-05-21T17:41:33.197958+0000","last_unstale":"2024-05-21T17:41:34.198049+0000","last_undegraded":"2024-05-21T17:41:34.198049+0000","last_fullsized":"2024-05-21T17:41:34.198049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:06:55.657012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"15'8","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196258+0000","last_change":"2024-05-21T17:41:32.194159+0000","last_active":"2024-05-21T17:41:34.196258+0000","last_peered":"2024-05-21T17:41:34.196258+0000","last_clean":"2024-05-21T17:41:34.196258+0000","last_became_active":"2024-05-21T17:41:32.193896+0000","last_became_peered":"2024-05-21T17:41:32.193896+0000","last_unstale":"2024-05-21T17:41:34.196258+0000","last_undegraded":"2024-05-21T17:41:34.196258+0000","last_fullsized":"2024-05-21T17:41:34.196258+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:31:47.729777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181569+0000","last_change":"2024-05-21T17:41:32.186754+0000","last_active":"2024-05-21T17:41:35.181569+0000","last_peered":"2024-05-21T17:41:35.181569+0000","last_clean":"2024-05-21T17:41:35.181569+0000","last_became_active":"2024-05-21T17:41:32.186477+0000","last_became_peered":"2024-05-21T17:41:32.186477+0000","last_unstale":"2024-05-21T17:41:35.181569+0000","last_undegraded":"2024-05-21T17:41:35.181569+0000","last_fullsized":"2024-05-21T17:41:35.181569+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:43:03.647293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503942+0000","last_change":"2024-05-21T17:41:33.214302+0000","last_active":"2024-05-21T17:41:35.503942+0000","last_peered":"2024-05-21T17:41:35.503942+0000","last_clean":"2024-05-21T17:41:35.503942+0000","last_became_active":"2024-05-21T17:41:33.211621+0000","last_became_peered":"2024-05-21T17:41:33.211621+0000","last_unstale":"2024-05-21T17:41:35.503942+0000","last_undegraded":"2024-05-21T17:41:35.503942+0000","last_fullsized":"2024-05-21T17:41:35.503942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:29.332651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181671+0000","last_change":"2024-05-21T17:41:32.185517+0000","last_active":"2024-05-21T17:41:35.181671+0000","last_peered":"2024-05-21T17:41:35.181671+0000","last_clean":"2024-05-21T17:41:35.181671+0000","last_became_active":"2024-05-21T17:41:32.185147+0000","last_became_peered":"2024-05-21T17:41:32.185147+0000","last_unstale":"2024-05-21T17:41:35.181671+0000","last_undegraded":"2024-05-21T17:41:35.181671+0000","last_fullsized":"2024-05-21T17:41:35.181671+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:51:23.199039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195591+0000","last_change":"2024-05-21T17:41:33.192101+0000","last_active":"2024-05-21T17:41:34.195591+0000","last_peered":"2024-05-21T17:41:34.195591+0000","last_clean":"2024-05-21T17:41:34.195591+0000","last_became_active":"2024-05-21T17:41:33.191796+0000","last_became_peered":"2024-05-21T17:41:33.191796+0000","last_unstale":"2024-05-21T17:41:34.195591+0000","last_undegraded":"2024-05-21T17:41:34.195591+0000","last_fullsized":"2024-05-21T17:41:34.195591+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:03:00.661590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505135+0000","last_change":"2024-05-21T17:41:32.197897+0000","last_active":"2024-05-21T17:41:35.505135+0000","last_peered":"2024-05-21T17:41:35.505135+0000","last_clean":"2024-05-21T17:41:35.505135+0000","last_became_active":"2024-05-21T17:41:32.197609+0000","last_became_peered":"2024-05-21T17:41:32.197609+0000","last_unstale":"2024-05-21T17:41:35.505135+0000","last_undegraded":"2024-05-21T17:41:35.505135+0000","last_fullsized":"2024-05-21T17:41:35.505135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:29:06.885892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181127+0000","last_change":"2024-05-21T17:41:33.213002+0000","last_active":"2024-05-21T17:41:35.181127+0000","last_peered":"2024-05-21T17:41:35.181127+0000","last_clean":"2024-05-21T17:41:35.181127+0000","last_became_active":"2024-05-21T17:41:33.212695+0000","last_became_peered":"2024-05-21T17:41:33.212695+0000","last_unstale":"2024-05-21T17:41:35.181127+0000","last_undegraded":"2024-05-21T17:41:35.181127+0000","last_fullsized":"2024-05-21T17:41:35.181127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:48.164464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504834+0000","last_change":"2024-05-21T17:41:32.197048+0000","last_active":"2024-05-21T17:41:35.504834+0000","last_peered":"2024-05-21T17:41:35.504834+0000","last_clean":"2024-05-21T17:41:35.504834+0000","last_became_active":"2024-05-21T17:41:32.196772+0000","last_became_peered":"2024-05-21T17:41:32.196772+0000","last_unstale":"2024-05-21T17:41:35.504834+0000","last_undegraded":"2024-05-21T17:41:35.504834+0000","last_fullsized":"2024-05-21T17:41:35.504834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:40:57.685104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181361+0000","last_change":"2024-05-21T17:41:33.195970+0000","last_active":"2024-05-21T17:41:35.181361+0000","last_peered":"2024-05-21T17:41:35.181361+0000","last_clean":"2024-05-21T17:41:35.181361+0000","last_became_active":"2024-05-21T17:41:33.194971+0000","last_became_peered":"2024-05-21T17:41:33.194971+0000","last_unstale":"2024-05-21T17:41:35.181361+0000","last_undegraded":"2024-05-21T17:41:35.181361+0000","last_fullsized":"2024-05-21T17:41:35.181361+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:06:55.657012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198017+0000","last_change":"2024-05-21T17:41:32.184754+0000","last_active":"2024-05-21T17:41:34.198017+0000","last_peered":"2024-05-21T17:41:34.198017+0000","last_clean":"2024-05-21T17:41:34.198017+0000","last_became_active":"2024-05-21T17:41:32.184457+0000","last_became_peered":"2024-05-21T17:41:32.184457+0000","last_unstale":"2024-05-21T17:41:34.198017+0000","last_undegraded":"2024-05-21T17:41:34.198017+0000","last_fullsized":"2024-05-21T17:41:34.198017+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:33.195952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196321+0000","last_change":"2024-05-21T17:41:33.205458+0000","last_active":"2024-05-21T17:41:34.196321+0000","last_peered":"2024-05-21T17:41:34.196321+0000","last_clean":"2024-05-21T17:41:34.196321+0000","last_became_active":"2024-05-21T17:41:33.204911+0000","last_became_peered":"2024-05-21T17:41:33.204911+0000","last_unstale":"2024-05-21T17:41:34.196321+0000","last_undegraded":"2024-05-21T17:41:34.196321+0000","last_fullsized":"2024-05-21T17:41:34.196321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:37:23.705355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504432+0000","last_change":"2024-05-21T17:41:32.197569+0000","last_active":"2024-05-21T17:41:35.504432+0000","last_peered":"2024-05-21T17:41:35.504432+0000","last_clean":"2024-05-21T17:41:35.504432+0000","last_became_active":"2024-05-21T17:41:32.197242+0000","last_became_peered":"2024-05-21T17:41:32.197242+0000","last_unstale":"2024-05-21T17:41:35.504432+0000","last_undegraded":"2024-05-21T17:41:35.504432+0000","last_fullsized":"2024-05-21T17:41:35.504432+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:54:26.900686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504792+0000","last_change":"2024-05-21T17:41:33.217022+0000","last_active":"2024-05-21T17:41:35.504792+0000","last_peered":"2024-05-21T17:41:35.504792+0000","last_clean":"2024-05-21T17:41:35.504792+0000","last_became_active":"2024-05-21T17:41:33.216705+0000","last_became_peered":"2024-05-21T17:41:33.216705+0000","last_unstale":"2024-05-21T17:41:35.504792+0000","last_undegraded":"2024-05-21T17:41:35.504792+0000","last_fullsized":"2024-05-21T17:41:35.504792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:25:38.189640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504345+0000","last_change":"2024-05-21T17:41:32.194912+0000","last_active":"2024-05-21T17:41:35.504345+0000","last_peered":"2024-05-21T17:41:35.504345+0000","last_clean":"2024-05-21T17:41:35.504345+0000","last_became_active":"2024-05-21T17:41:32.194624+0000","last_became_peered":"2024-05-21T17:41:32.194624+0000","last_unstale":"2024-05-21T17:41:35.504345+0000","last_undegraded":"2024-05-21T17:41:35.504345+0000","last_fullsized":"2024-05-21T17:41:35.504345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:52:59.378552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181533+0000","last_change":"2024-05-21T17:41:33.210044+0000","last_active":"2024-05-21T17:41:35.181533+0000","last_peered":"2024-05-21T17:41:35.181533+0000","last_clean":"2024-05-21T17:41:35.181533+0000","last_became_active":"2024-05-21T17:41:33.209722+0000","last_became_peered":"2024-05-21T17:41:33.209722+0000","last_unstale":"2024-05-21T17:41:35.181533+0000","last_undegraded":"2024-05-21T17:41:35.181533+0000","last_fullsized":"2024-05-21T17:41:35.181533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:37:09.553771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504190+0000","last_change":"2024-05-21T17:41:32.195112+0000","last_active":"2024-05-21T17:41:35.504190+0000","last_peered":"2024-05-21T17:41:35.504190+0000","last_clean":"2024-05-21T17:41:35.504190+0000","last_became_active":"2024-05-21T17:41:32.194765+0000","last_became_peered":"2024-05-21T17:41:32.194765+0000","last_unstale":"2024-05-21T17:41:35.504190+0000","last_undegraded":"2024-05-21T17:41:35.504190+0000","last_fullsized":"2024-05-21T17:41:35.504190+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:11:23.430522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196636+0000","last_change":"2024-05-21T17:41:33.210126+0000","last_active":"2024-05-21T17:41:34.196636+0000","last_peered":"2024-05-21T17:41:34.196636+0000","last_clean":"2024-05-21T17:41:34.196636+0000","last_became_active":"2024-05-21T17:41:33.209679+0000","last_became_peered":"2024-05-21T17:41:33.209679+0000","last_unstale":"2024-05-21T17:41:34.196636+0000","last_undegraded":"2024-05-21T17:41:34.196636+0000","last_fullsized":"2024-05-21T17:41:34.196636+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:14:16.199161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":110,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196448+0000","last_change":"2024-05-21T17:41:27.156765+0000","last_active":"2024-05-21T17:41:34.196448+0000","last_peered":"2024-05-21T17:41:34.196448+0000","last_clean":"2024-05-21T17:41:34.196448+0000","last_became_active":"2024-05-21T17:41:27.156385+0000","last_became_peered":"2024-05-21T17:41:27.156385+0000","last_unstale":"2024-05-21T17:41:34.196448+0000","last_undegraded":"2024-05-21T17:41:34.196448+0000","last_fullsized":"2024-05-21T17:41:34.196448+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504752+0000","last_change":"2024-05-21T17:41:27.156565+0000","last_active":"2024-05-21T17:41:35.504752+0000","last_peered":"2024-05-21T17:41:35.504752+0000","last_clean":"2024-05-21T17:41:35.504752+0000","last_became_active":"2024-05-21T17:41:27.156230+0000","last_became_peered":"2024-05-21T17:41:27.156230+0000","last_unstale":"2024-05-21T17:41:35.504752+0000","last_undegraded":"2024-05-21T17:41:35.504752+0000","last_fullsized":"2024-05-21T17:41:35.504752+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:27.800826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504635+0000","last_change":"2024-05-21T17:41:32.196927+0000","last_active":"2024-05-21T17:41:35.504635+0000","last_peered":"2024-05-21T17:41:35.504635+0000","last_clean":"2024-05-21T17:41:35.504635+0000","last_became_active":"2024-05-21T17:41:32.196462+0000","last_became_peered":"2024-05-21T17:41:32.196462+0000","last_unstale":"2024-05-21T17:41:35.504635+0000","last_undegraded":"2024-05-21T17:41:35.504635+0000","last_fullsized":"2024-05-21T17:41:35.504635+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:47.159182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505094+0000","last_change":"2024-05-21T17:41:33.214046+0000","last_active":"2024-05-21T17:41:35.505094+0000","last_peered":"2024-05-21T17:41:35.505094+0000","last_clean":"2024-05-21T17:41:35.505094+0000","last_became_active":"2024-05-21T17:41:33.213672+0000","last_became_peered":"2024-05-21T17:41:33.213672+0000","last_unstale":"2024-05-21T17:41:35.505094+0000","last_undegraded":"2024-05-21T17:41:35.505094+0000","last_fullsized":"2024-05-21T17:41:35.505094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:12.064652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503362+0000","last_change":"2024-05-21T17:41:27.155004+0000","last_active":"2024-05-21T17:41:35.503362+0000","last_peered":"2024-05-21T17:41:35.503362+0000","last_clean":"2024-05-21T17:41:35.503362+0000","last_became_active":"2024-05-21T17:41:27.154231+0000","last_became_peered":"2024-05-21T17:41:27.154231+0000","last_unstale":"2024-05-21T17:41:35.503362+0000","last_undegraded":"2024-05-21T17:41:35.503362+0000","last_fullsized":"2024-05-21T17:41:35.503362+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:16:06.123434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197552+0000","last_change":"2024-05-21T17:41:32.177330+0000","last_active":"2024-05-21T17:41:34.197552+0000","last_peered":"2024-05-21T17:41:34.197552+0000","last_clean":"2024-05-21T17:41:34.197552+0000","last_became_active":"2024-05-21T17:41:32.176953+0000","last_became_peered":"2024-05-21T17:41:32.176953+0000","last_unstale":"2024-05-21T17:41:34.197552+0000","last_undegraded":"2024-05-21T17:41:34.197552+0000","last_fullsized":"2024-05-21T17:41:34.197552+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:58:39.302037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504924+0000","last_change":"2024-05-21T17:41:33.203272+0000","last_active":"2024-05-21T17:41:35.504924+0000","last_peered":"2024-05-21T17:41:35.504924+0000","last_clean":"2024-05-21T17:41:35.504924+0000","last_became_active":"2024-05-21T17:41:33.202730+0000","last_became_peered":"2024-05-21T17:41:33.202730+0000","last_unstale":"2024-05-21T17:41:35.504924+0000","last_undegraded":"2024-05-21T17:41:35.504924+0000","last_fullsized":"2024-05-21T17:41:35.504924+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:01:35.849261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.180459+0000","last_change":"2024-05-21T17:41:27.159363+0000","last_active":"2024-05-21T17:41:35.180459+0000","last_peered":"2024-05-21T17:41:35.180459+0000","last_clean":"2024-05-21T17:41:35.180459+0000","last_became_active":"2024-05-21T17:41:27.159108+0000","last_became_peered":"2024-05-21T17:41:27.159108+0000","last_unstale":"2024-05-21T17:41:35.180459+0000","last_undegraded":"2024-05-21T17:41:35.180459+0000","last_fullsized":"2024-05-21T17:41:35.180459+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197258+0000","last_change":"2024-05-21T17:41:32.176351+0000","last_active":"2024-05-21T17:41:34.197258+0000","last_peered":"2024-05-21T17:41:34.197258+0000","last_clean":"2024-05-21T17:41:34.197258+0000","last_became_active":"2024-05-21T17:41:32.176008+0000","last_became_peered":"2024-05-21T17:41:32.176008+0000","last_unstale":"2024-05-21T17:41:34.197258+0000","last_undegraded":"2024-05-21T17:41:34.197258+0000","last_fullsized":"2024-05-21T17:41:34.197258+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:07:22.267186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198105+0000","last_change":"2024-05-21T17:41:33.198808+0000","last_active":"2024-05-21T17:41:34.198105+0000","last_peered":"2024-05-21T17:41:34.198105+0000","last_clean":"2024-05-21T17:41:34.198105+0000","last_became_active":"2024-05-21T17:41:33.197653+0000","last_became_peered":"2024-05-21T17:41:33.197653+0000","last_unstale":"2024-05-21T17:41:34.198105+0000","last_undegraded":"2024-05-21T17:41:34.198105+0000","last_fullsized":"2024-05-21T17:41:34.198105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:40:32.331027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503358+0000","last_change":"2024-05-21T17:41:27.154942+0000","last_active":"2024-05-21T17:41:35.503358+0000","last_peered":"2024-05-21T17:41:35.503358+0000","last_clean":"2024-05-21T17:41:35.503358+0000","last_became_active":"2024-05-21T17:41:27.154055+0000","last_became_peered":"2024-05-21T17:41:27.154055+0000","last_unstale":"2024-05-21T17:41:35.503358+0000","last_undegraded":"2024-05-21T17:41:35.503358+0000","last_fullsized":"2024-05-21T17:41:35.503358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:58:33.630777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181371+0000","last_change":"2024-05-21T17:41:32.177942+0000","last_active":"2024-05-21T17:41:35.181371+0000","last_peered":"2024-05-21T17:41:35.181371+0000","last_clean":"2024-05-21T17:41:35.181371+0000","last_became_active":"2024-05-21T17:41:32.177474+0000","last_became_peered":"2024-05-21T17:41:32.177474+0000","last_unstale":"2024-05-21T17:41:35.181371+0000","last_undegraded":"2024-05-21T17:41:35.181371+0000","last_fullsized":"2024-05-21T17:41:35.181371+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:26:32.625845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198276+0000","last_change":"2024-05-21T17:41:33.196975+0000","last_active":"2024-05-21T17:41:34.198276+0000","last_peered":"2024-05-21T17:41:34.198276+0000","last_clean":"2024-05-21T17:41:34.198276+0000","last_became_active":"2024-05-21T17:41:33.196465+0000","last_became_peered":"2024-05-21T17:41:33.196465+0000","last_unstale":"2024-05-21T17:41:34.198276+0000","last_undegraded":"2024-05-21T17:41:34.198276+0000","last_fullsized":"2024-05-21T17:41:34.198276+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:12:57.454391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181373+0000","last_change":"2024-05-21T17:41:32.177494+0000","last_active":"2024-05-21T17:41:35.181373+0000","last_peered":"2024-05-21T17:41:35.181373+0000","last_clean":"2024-05-21T17:41:35.181373+0000","last_became_active":"2024-05-21T17:41:32.177206+0000","last_became_peered":"2024-05-21T17:41:32.177206+0000","last_unstale":"2024-05-21T17:41:35.181373+0000","last_undegraded":"2024-05-21T17:41:35.181373+0000","last_fullsized":"2024-05-21T17:41:35.181373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:58:39.302037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198316+0000","last_change":"2024-05-21T17:41:27.154907+0000","last_active":"2024-05-21T17:41:34.198316+0000","last_peered":"2024-05-21T17:41:34.198316+0000","last_clean":"2024-05-21T17:41:34.198316+0000","last_became_active":"2024-05-21T17:41:27.154659+0000","last_became_peered":"2024-05-21T17:41:27.154659+0000","last_unstale":"2024-05-21T17:41:34.198316+0000","last_undegraded":"2024-05-21T17:41:34.198316+0000","last_fullsized":"2024-05-21T17:41:34.198316+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:28:47.634110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195969+0000","last_change":"2024-05-21T17:41:33.210001+0000","last_active":"2024-05-21T17:41:34.195969+0000","last_peered":"2024-05-21T17:41:34.195969+0000","last_clean":"2024-05-21T17:41:34.195969+0000","last_became_active":"2024-05-21T17:41:33.209694+0000","last_became_peered":"2024-05-21T17:41:33.209694+0000","last_unstale":"2024-05-21T17:41:34.195969+0000","last_undegraded":"2024-05-21T17:41:34.195969+0000","last_fullsized":"2024-05-21T17:41:34.195969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:29:52.943757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198249+0000","last_change":"2024-05-21T17:41:32.177540+0000","last_active":"2024-05-21T17:41:34.198249+0000","last_peered":"2024-05-21T17:41:34.198249+0000","last_clean":"2024-05-21T17:41:34.198249+0000","last_became_active":"2024-05-21T17:41:32.177253+0000","last_became_peered":"2024-05-21T17:41:32.177253+0000","last_unstale":"2024-05-21T17:41:34.198249+0000","last_undegraded":"2024-05-21T17:41:34.198249+0000","last_fullsized":"2024-05-21T17:41:34.198249+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:52.250548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503386+0000","last_change":"2024-05-21T17:41:27.154725+0000","last_active":"2024-05-21T17:41:35.503386+0000","last_peered":"2024-05-21T17:41:35.503386+0000","last_clean":"2024-05-21T17:41:35.503386+0000","last_became_active":"2024-05-21T17:41:27.154339+0000","last_became_peered":"2024-05-21T17:41:27.154339+0000","last_unstale":"2024-05-21T17:41:35.503386+0000","last_undegraded":"2024-05-21T17:41:35.503386+0000","last_fullsized":"2024-05-21T17:41:35.503386+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:47:27.935607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196199+0000","last_change":"2024-05-21T17:41:33.206242+0000","last_active":"2024-05-21T17:41:34.196199+0000","last_peered":"2024-05-21T17:41:34.196199+0000","last_clean":"2024-05-21T17:41:34.196199+0000","last_became_active":"2024-05-21T17:41:33.205945+0000","last_became_peered":"2024-05-21T17:41:33.205945+0000","last_unstale":"2024-05-21T17:41:34.196199+0000","last_undegraded":"2024-05-21T17:41:34.196199+0000","last_fullsized":"2024-05-21T17:41:34.196199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:36:01.886190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505021+0000","last_change":"2024-05-21T17:41:32.193781+0000","last_active":"2024-05-21T17:41:35.505021+0000","last_peered":"2024-05-21T17:41:35.505021+0000","last_clean":"2024-05-21T17:41:35.505021+0000","last_became_active":"2024-05-21T17:41:32.193464+0000","last_became_peered":"2024-05-21T17:41:32.193464+0000","last_unstale":"2024-05-21T17:41:35.505021+0000","last_undegraded":"2024-05-21T17:41:35.505021+0000","last_fullsized":"2024-05-21T17:41:35.505021+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:29:25.307585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198117+0000","last_change":"2024-05-21T17:41:27.156568+0000","last_active":"2024-05-21T17:41:34.198117+0000","last_peered":"2024-05-21T17:41:34.198117+0000","last_clean":"2024-05-21T17:41:34.198117+0000","last_became_active":"2024-05-21T17:41:27.156293+0000","last_became_peered":"2024-05-21T17:41:27.156293+0000","last_unstale":"2024-05-21T17:41:34.198117+0000","last_undegraded":"2024-05-21T17:41:34.198117+0000","last_fullsized":"2024-05-21T17:41:34.198117+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503989+0000","last_change":"2024-05-21T17:41:33.215019+0000","last_active":"2024-05-21T17:41:35.503989+0000","last_peered":"2024-05-21T17:41:35.503989+0000","last_clean":"2024-05-21T17:41:35.503989+0000","last_became_active":"2024-05-21T17:41:33.214714+0000","last_became_peered":"2024-05-21T17:41:33.214714+0000","last_unstale":"2024-05-21T17:41:35.503989+0000","last_undegraded":"2024-05-21T17:41:35.503989+0000","last_fullsized":"2024-05-21T17:41:35.503989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:02:10.347399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.179976+0000","last_change":"2024-05-21T17:41:32.185252+0000","last_active":"2024-05-21T17:41:35.179976+0000","last_peered":"2024-05-21T17:41:35.179976+0000","last_clean":"2024-05-21T17:41:35.179976+0000","last_became_active":"2024-05-21T17:41:32.184893+0000","last_became_peered":"2024-05-21T17:41:32.184893+0000","last_unstale":"2024-05-21T17:41:35.179976+0000","last_undegraded":"2024-05-21T17:41:35.179976+0000","last_fullsized":"2024-05-21T17:41:35.179976+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:33.195952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503430+0000","last_change":"2024-05-21T17:41:27.156830+0000","last_active":"2024-05-21T17:41:35.503430+0000","last_peered":"2024-05-21T17:41:35.503430+0000","last_clean":"2024-05-21T17:41:35.503430+0000","last_became_active":"2024-05-21T17:41:27.156384+0000","last_became_peered":"2024-05-21T17:41:27.156384+0000","last_unstale":"2024-05-21T17:41:35.503430+0000","last_undegraded":"2024-05-21T17:41:35.503430+0000","last_fullsized":"2024-05-21T17:41:35.503430+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:21:15.563350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.180996+0000","last_change":"2024-05-21T17:41:33.197551+0000","last_active":"2024-05-21T17:41:35.180996+0000","last_peered":"2024-05-21T17:41:35.180996+0000","last_clean":"2024-05-21T17:41:35.180996+0000","last_became_active":"2024-05-21T17:41:33.196604+0000","last_became_peered":"2024-05-21T17:41:33.196604+0000","last_unstale":"2024-05-21T17:41:35.180996+0000","last_undegraded":"2024-05-21T17:41:35.180996+0000","last_fullsized":"2024-05-21T17:41:35.180996+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:11:24.435803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181240+0000","last_change":"2024-05-21T17:41:33.210151+0000","last_active":"2024-05-21T17:41:35.181240+0000","last_peered":"2024-05-21T17:41:35.181240+0000","last_clean":"2024-05-21T17:41:35.181240+0000","last_became_active":"2024-05-21T17:41:33.209322+0000","last_became_peered":"2024-05-21T17:41:33.209322+0000","last_unstale":"2024-05-21T17:41:35.181240+0000","last_undegraded":"2024-05-21T17:41:35.181240+0000","last_fullsized":"2024-05-21T17:41:35.181240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:29:07.891173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"15'6","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504664+0000","last_change":"2024-05-21T17:41:32.197388+0000","last_active":"2024-05-21T17:41:35.504664+0000","last_peered":"2024-05-21T17:41:35.504664+0000","last_clean":"2024-05-21T17:41:35.504664+0000","last_became_active":"2024-05-21T17:41:32.196445+0000","last_became_peered":"2024-05-21T17:41:32.196445+0000","last_unstale":"2024-05-21T17:41:35.504664+0000","last_undegraded":"2024-05-21T17:41:35.504664+0000","last_fullsized":"2024-05-21T17:41:35.504664+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:08:36.015445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196326+0000","last_change":"2024-05-21T17:41:33.215210+0000","last_active":"2024-05-21T17:41:34.196326+0000","last_peered":"2024-05-21T17:41:34.196326+0000","last_clean":"2024-05-21T17:41:34.196326+0000","last_became_active":"2024-05-21T17:41:33.214780+0000","last_became_peered":"2024-05-21T17:41:33.214780+0000","last_unstale":"2024-05-21T17:41:34.196326+0000","last_undegraded":"2024-05-21T17:41:34.196326+0000","last_fullsized":"2024-05-21T17:41:34.196326+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:32:22.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"15'3","reported_seq":17,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181189+0000","last_change":"2024-05-21T17:41:32.190183+0000","last_active":"2024-05-21T17:41:35.181189+0000","last_peered":"2024-05-21T17:41:35.181189+0000","last_clean":"2024-05-21T17:41:35.181189+0000","last_became_active":"2024-05-21T17:41:32.189858+0000","last_became_peered":"2024-05-21T17:41:32.189858+0000","last_unstale":"2024-05-21T17:41:35.181189+0000","last_undegraded":"2024-05-21T17:41:35.181189+0000","last_fullsized":"2024-05-21T17:41:35.181189+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:06:50.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181637+0000","last_change":"2024-05-21T17:41:32.183393+0000","last_active":"2024-05-21T17:41:35.181637+0000","last_peered":"2024-05-21T17:41:35.181637+0000","last_clean":"2024-05-21T17:41:35.181637+0000","last_became_active":"2024-05-21T17:41:32.183094+0000","last_became_peered":"2024-05-21T17:41:32.183094+0000","last_unstale":"2024-05-21T17:41:35.181637+0000","last_undegraded":"2024-05-21T17:41:35.181637+0000","last_fullsized":"2024-05-21T17:41:35.181637+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:31:47.729777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197469+0000","last_change":"2024-05-21T17:41:33.203119+0000","last_active":"2024-05-21T17:41:34.197469+0000","last_peered":"2024-05-21T17:41:34.197469+0000","last_clean":"2024-05-21T17:41:34.197469+0000","last_became_active":"2024-05-21T17:41:33.202860+0000","last_became_peered":"2024-05-21T17:41:33.202860+0000","last_unstale":"2024-05-21T17:41:34.197469+0000","last_undegraded":"2024-05-21T17:41:34.197469+0000","last_fullsized":"2024-05-21T17:41:34.197469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:07:06.410067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196732+0000","last_change":"2024-05-21T17:41:32.192104+0000","last_active":"2024-05-21T17:41:34.196732+0000","last_peered":"2024-05-21T17:41:34.196732+0000","last_clean":"2024-05-21T17:41:34.196732+0000","last_became_active":"2024-05-21T17:41:32.191411+0000","last_became_peered":"2024-05-21T17:41:32.191411+0000","last_unstale":"2024-05-21T17:41:34.196732+0000","last_undegraded":"2024-05-21T17:41:34.196732+0000","last_fullsized":"2024-05-21T17:41:34.196732+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:43:03.647293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503768+0000","last_change":"2024-05-21T17:41:33.212185+0000","last_active":"2024-05-21T17:41:35.503768+0000","last_peered":"2024-05-21T17:41:35.503768+0000","last_clean":"2024-05-21T17:41:35.503768+0000","last_became_active":"2024-05-21T17:41:33.211214+0000","last_became_peered":"2024-05-21T17:41:33.211214+0000","last_unstale":"2024-05-21T17:41:35.503768+0000","last_undegraded":"2024-05-21T17:41:35.503768+0000","last_fullsized":"2024-05-21T17:41:35.503768+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:03:55.087212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504945+0000","last_change":"2024-05-21T17:41:33.206294+0000","last_active":"2024-05-21T17:41:35.504945+0000","last_peered":"2024-05-21T17:41:35.504945+0000","last_clean":"2024-05-21T17:41:35.504945+0000","last_became_active":"2024-05-21T17:41:33.205960+0000","last_became_peered":"2024-05-21T17:41:33.205960+0000","last_unstale":"2024-05-21T17:41:35.504945+0000","last_undegraded":"2024-05-21T17:41:35.504945+0000","last_fullsized":"2024-05-21T17:41:35.504945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:06.686603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504088+0000","last_change":"2024-05-21T17:41:32.198839+0000","last_active":"2024-05-21T17:41:35.504088+0000","last_peered":"2024-05-21T17:41:35.504088+0000","last_clean":"2024-05-21T17:41:35.504088+0000","last_became_active":"2024-05-21T17:41:32.198523+0000","last_became_peered":"2024-05-21T17:41:32.198523+0000","last_unstale":"2024-05-21T17:41:35.504088+0000","last_undegraded":"2024-05-21T17:41:35.504088+0000","last_fullsized":"2024-05-21T17:41:35.504088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:07:05.404785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197963+0000","last_change":"2024-05-21T17:41:33.186217+0000","last_active":"2024-05-21T17:41:34.197963+0000","last_peered":"2024-05-21T17:41:34.197963+0000","last_clean":"2024-05-21T17:41:34.197963+0000","last_became_active":"2024-05-21T17:41:33.185935+0000","last_became_peered":"2024-05-21T17:41:33.185935+0000","last_unstale":"2024-05-21T17:41:34.197963+0000","last_undegraded":"2024-05-21T17:41:34.197963+0000","last_fullsized":"2024-05-21T17:41:34.197963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:00:42.905885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195682+0000","last_change":"2024-05-21T17:41:32.191785+0000","last_active":"2024-05-21T17:41:34.195682+0000","last_peered":"2024-05-21T17:41:34.195682+0000","last_clean":"2024-05-21T17:41:34.195682+0000","last_became_active":"2024-05-21T17:41:32.190999+0000","last_became_peered":"2024-05-21T17:41:32.190999+0000","last_unstale":"2024-05-21T17:41:34.195682+0000","last_undegraded":"2024-05-21T17:41:34.195682+0000","last_fullsized":"2024-05-21T17:41:34.195682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:52.250548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504863+0000","last_change":"2024-05-21T17:41:33.213982+0000","last_active":"2024-05-21T17:41:35.504863+0000","last_peered":"2024-05-21T17:41:35.504863+0000","last_clean":"2024-05-21T17:41:35.504863+0000","last_became_active":"2024-05-21T17:41:33.213225+0000","last_became_peered":"2024-05-21T17:41:33.213225+0000","last_unstale":"2024-05-21T17:41:35.504863+0000","last_undegraded":"2024-05-21T17:41:35.504863+0000","last_fullsized":"2024-05-21T17:41:35.504863+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:07:15.129363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196466+0000","last_change":"2024-05-21T17:41:32.182693+0000","last_active":"2024-05-21T17:41:34.196466+0000","last_peered":"2024-05-21T17:41:34.196466+0000","last_clean":"2024-05-21T17:41:34.196466+0000","last_became_active":"2024-05-21T17:41:32.182420+0000","last_became_peered":"2024-05-21T17:41:32.182420+0000","last_unstale":"2024-05-21T17:41:34.196466+0000","last_undegraded":"2024-05-21T17:41:34.196466+0000","last_fullsized":"2024-05-21T17:41:34.196466+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:26:32.625845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198167+0000","last_change":"2024-05-21T17:41:33.198020+0000","last_active":"2024-05-21T17:41:34.198167+0000","last_peered":"2024-05-21T17:41:34.198167+0000","last_clean":"2024-05-21T17:41:34.198167+0000","last_became_active":"2024-05-21T17:41:33.197187+0000","last_became_peered":"2024-05-21T17:41:33.197187+0000","last_unstale":"2024-05-21T17:41:34.198167+0000","last_undegraded":"2024-05-21T17:41:34.198167+0000","last_fullsized":"2024-05-21T17:41:34.198167+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:37:09.553771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504321+0000","last_change":"2024-05-21T17:41:32.197952+0000","last_active":"2024-05-21T17:41:35.504321+0000","last_peered":"2024-05-21T17:41:35.504321+0000","last_clean":"2024-05-21T17:41:35.504321+0000","last_became_active":"2024-05-21T17:41:32.197316+0000","last_became_peered":"2024-05-21T17:41:32.197316+0000","last_unstale":"2024-05-21T17:41:35.504321+0000","last_undegraded":"2024-05-21T17:41:35.504321+0000","last_fullsized":"2024-05-21T17:41:35.504321+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:04:40.497094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181105+0000","last_change":"2024-05-21T17:41:33.196406+0000","last_active":"2024-05-21T17:41:35.181105+0000","last_peered":"2024-05-21T17:41:35.181105+0000","last_clean":"2024-05-21T17:41:35.181105+0000","last_became_active":"2024-05-21T17:41:33.195986+0000","last_became_peered":"2024-05-21T17:41:33.195986+0000","last_unstale":"2024-05-21T17:41:35.181105+0000","last_undegraded":"2024-05-21T17:41:35.181105+0000","last_fullsized":"2024-05-21T17:41:35.181105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:54:27.905968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196455+0000","last_change":"2024-05-21T17:41:32.192119+0000","last_active":"2024-05-21T17:41:34.196455+0000","last_peered":"2024-05-21T17:41:34.196455+0000","last_clean":"2024-05-21T17:41:34.196455+0000","last_became_active":"2024-05-21T17:41:32.191275+0000","last_became_peered":"2024-05-21T17:41:32.191275+0000","last_unstale":"2024-05-21T17:41:34.196455+0000","last_undegraded":"2024-05-21T17:41:34.196455+0000","last_fullsized":"2024-05-21T17:41:34.196455+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:58:39.302037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181217+0000","last_change":"2024-05-21T17:41:33.196519+0000","last_active":"2024-05-21T17:41:35.181217+0000","last_peered":"2024-05-21T17:41:35.181217+0000","last_clean":"2024-05-21T17:41:35.181217+0000","last_became_active":"2024-05-21T17:41:33.194635+0000","last_became_peered":"2024-05-21T17:41:33.194635+0000","last_unstale":"2024-05-21T17:41:35.181217+0000","last_undegraded":"2024-05-21T17:41:35.181217+0000","last_fullsized":"2024-05-21T17:41:35.181217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:00:42.905885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505022+0000","last_change":"2024-05-21T17:41:32.198015+0000","last_active":"2024-05-21T17:41:35.505022+0000","last_peered":"2024-05-21T17:41:35.505022+0000","last_clean":"2024-05-21T17:41:35.505022+0000","last_became_active":"2024-05-21T17:41:32.197483+0000","last_became_peered":"2024-05-21T17:41:32.197483+0000","last_unstale":"2024-05-21T17:41:35.505022+0000","last_undegraded":"2024-05-21T17:41:35.505022+0000","last_fullsized":"2024-05-21T17:41:35.505022+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:00:41.900604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504078+0000","last_change":"2024-05-21T17:41:33.207480+0000","last_active":"2024-05-21T17:41:35.504078+0000","last_peered":"2024-05-21T17:41:35.504078+0000","last_clean":"2024-05-21T17:41:35.504078+0000","last_became_active":"2024-05-21T17:41:33.207211+0000","last_became_peered":"2024-05-21T17:41:33.207211+0000","last_unstale":"2024-05-21T17:41:35.504078+0000","last_undegraded":"2024-05-21T17:41:35.504078+0000","last_fullsized":"2024-05-21T17:41:35.504078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:55:50.762859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198168+0000","last_change":"2024-05-21T17:41:32.180196+0000","last_active":"2024-05-21T17:41:34.198168+0000","last_peered":"2024-05-21T17:41:34.198168+0000","last_clean":"2024-05-21T17:41:34.198168+0000","last_became_active":"2024-05-21T17:41:32.179887+0000","last_became_peered":"2024-05-21T17:41:32.179887+0000","last_unstale":"2024-05-21T17:41:34.198168+0000","last_undegraded":"2024-05-21T17:41:34.198168+0000","last_fullsized":"2024-05-21T17:41:34.198168+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:06:50.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.180828+0000","last_change":"2024-05-21T17:41:33.196424+0000","last_active":"2024-05-21T17:41:35.180828+0000","last_peered":"2024-05-21T17:41:35.180828+0000","last_clean":"2024-05-21T17:41:35.180828+0000","last_became_active":"2024-05-21T17:41:33.196119+0000","last_became_peered":"2024-05-21T17:41:33.196119+0000","last_unstale":"2024-05-21T17:41:35.180828+0000","last_undegraded":"2024-05-21T17:41:35.180828+0000","last_fullsized":"2024-05-21T17:41:35.180828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:29:26.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"15'4","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196681+0000","last_change":"2024-05-21T17:41:32.182802+0000","last_active":"2024-05-21T17:41:34.196681+0000","last_peered":"2024-05-21T17:41:34.196681+0000","last_clean":"2024-05-21T17:41:34.196681+0000","last_became_active":"2024-05-21T17:41:32.182446+0000","last_became_peered":"2024-05-21T17:41:32.182446+0000","last_unstale":"2024-05-21T17:41:34.196681+0000","last_undegraded":"2024-05-21T17:41:34.196681+0000","last_fullsized":"2024-05-21T17:41:34.196681+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:06:50.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17264,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":47535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":642780,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":641696,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":51236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T17:41:37.609 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-21T17:41:37.610 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-21T17:41:37.610 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-21T17:41:37.610 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-21T17:41:37.610 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-21T17:41:37.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.746+0000 7fa96142a640 1 Processor -- start 2024-05-21T17:41:37.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.746+0000 7fa96142a640 1 -- start start 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.746+0000 7fa96142a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa95c0a1720 0x7fa95c0a2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.746+0000 7fa96142a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa95c14c7f0 0x7fa95c14ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.746+0000 7fa96142a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95c0a3150 con 0x7fa95c074750 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.746+0000 7fa96142a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fa95c0a2070 con 0x7fa95c14c7f0 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.746+0000 7fa96142a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa95c1453d0 con 0x7fa95c0a1720 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.746+0000 7fa95b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fa95c14c7f0 0x7fa95c14ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.746+0000 7fa95a7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa95c074750 0x7fa95c146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60674/0 (socket says 172.21.15.3:60674) 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.746+0000 7fa95a7fc640 1 -- 172.21.15.3:0/1313923745 learned_addr learned my addr 172.21.15.3:0/1313923745 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.747+0000 7fa95affd640 1 --2- 172.21.15.3:0/1313923745 >> v2:172.21.15.203:3300/0 conn(0x7fa95c0a1720 0x7fa95c0a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.747+0000 7fa95b7fe640 1 -- 172.21.15.3:0/1313923745 >> v2:172.21.15.203:3300/0 conn(0x7fa95c0a1720 msgr2=0x7fa95c0a2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.747+0000 7fa95b7fe640 1 --2- 172.21.15.3:0/1313923745 >> v2:172.21.15.203:3300/0 conn(0x7fa95c0a1720 0x7fa95c0a2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.747+0000 7fa95b7fe640 1 -- 172.21.15.3:0/1313923745 >> v1:172.21.15.3:6789/0 conn(0x7fa95c074750 legacy=0x7fa95c146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.747+0000 7fa959ffb640 1 -- 172.21.15.3:0/1313923745 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 783240497 0 0) 0x7fa95c0a3150 con 0x7fa95c074750 2024-05-21T17:41:37.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.747+0000 7fa95b7fe640 1 -- 172.21.15.3:0/1313923745 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c06ead0 con 0x7fa95c14c7f0 2024-05-21T17:41:37.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.747+0000 7fa95affd640 1 --2- 172.21.15.3:0/1313923745 >> v2:172.21.15.203:3300/0 conn(0x7fa95c0a1720 0x7fa95c0a2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:37.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa95b7fe640 1 --2- 172.21.15.3:0/1313923745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c14c7f0 0x7fa95c14ebe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0a7500 tx=0x7fa95402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593c34ac1a5c9fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:37.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa959ffb640 1 -- 172.21.15.3:0/1313923745 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954039070 con 0x7fa95c14c7f0 2024-05-21T17:41:37.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa959ffb640 1 -- 172.21.15.3:0/1313923745 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95402cbf0 con 0x7fa95c14c7f0 2024-05-21T17:41:37.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa959ffb640 1 -- 172.21.15.3:0/1313923745 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954034440 con 0x7fa95c14c7f0 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa96142a640 1 -- 172.21.15.3:0/1313923745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c14c7f0 msgr2=0x7fa95c14ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa96142a640 1 --2- 172.21.15.3:0/1313923745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c14c7f0 0x7fa95c14ebe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0a7500 tx=0x7fa95402be40 comp rx=0 tx=0).stop 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa96142a640 1 -- 172.21.15.3:0/1313923745 shutdown_connections 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa96142a640 1 --2- 172.21.15.3:0/1313923745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c14c7f0 0x7fa95c14ebe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa96142a640 1 --2- 172.21.15.3:0/1313923745 >> v2:172.21.15.203:3300/0 conn(0x7fa95c0a1720 0x7fa95c0a2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa96142a640 1 -- 172.21.15.3:0/1313923745 >> 172.21.15.3:0/1313923745 conn(0x7fa95c06ccf0 msgr2=0x7fa95c0a6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa96142a640 1 -- 172.21.15.3:0/1313923745 shutdown_connections 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.748+0000 7fa96142a640 1 -- 172.21.15.3:0/1313923745 wait complete. 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.749+0000 7fa96142a640 1 Processor -- start 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.749+0000 7fa96142a640 1 -- start start 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa96142a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c074750 0x7fa95c13c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa96142a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa95c0a1720 0x7fa95c13db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa96142a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa95c14c7f0 0x7fa95c0b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa96142a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c06e050 con 0x7fa95c074750 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa96142a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c0a2d90 con 0x7fa95c14c7f0 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa96142a640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fa95c13d6d0 con 0x7fa95c0a1720 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c074750 0x7fa95c13c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95a7fc640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa95c0a1720 0x7fa95c13db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95affd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c074750 0x7fa95c13c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51416/0 (socket says 172.21.15.3:51416) 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95affd640 1 -- 172.21.15.3:0/2366430713 learned_addr learned my addr 172.21.15.3:0/2366430713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95b7fe640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa95c14c7f0 0x7fa95c0b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95affd640 1 -- 172.21.15.3:0/2366430713 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa95c0a1720 msgr2=0x7fa95c13db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95affd640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa95c0a1720 0x7fa95c13db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95affd640 1 -- 172.21.15.3:0/2366430713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa95c14c7f0 msgr2=0x7fa95c0b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:37.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95affd640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa95c14c7f0 0x7fa95c0b7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95affd640 1 -- 172.21.15.3:0/2366430713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c072c90 con 0x7fa95c074750 2024-05-21T17:41:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.750+0000 7fa95a7fc640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa95c0a1720 0x7fa95c13db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.751+0000 7fa95affd640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c074750 0x7fa95c13c050 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00c7f0 tx=0x7fa94c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.751+0000 7fa9437fe640 1 -- 172.21.15.3:0/2366430713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c015070 con 0x7fa95c074750 2024-05-21T17:41:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.751+0000 7fa9437fe640 1 -- 172.21.15.3:0/2366430713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c007a30 con 0x7fa95c074750 2024-05-21T17:41:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.751+0000 7fa96142a640 1 -- 172.21.15.3:0/2366430713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95c13c690 con 0x7fa95c074750 2024-05-21T17:41:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.751+0000 7fa9437fe640 1 -- 172.21.15.3:0/2366430713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c007e00 con 0x7fa95c074750 2024-05-21T17:41:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.751+0000 7fa96142a640 1 -- 172.21.15.3:0/2366430713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95c0b7b80 con 0x7fa95c074750 2024-05-21T17:41:37.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.752+0000 7fa96142a640 1 -- 172.21.15.3:0/2366430713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95c069a90 con 0x7fa95c074750 2024-05-21T17:41:37.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.753+0000 7fa9437fe640 1 -- 172.21.15.3:0/2366430713 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7fa94c018070 con 0x7fa95c074750 2024-05-21T17:41:37.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.753+0000 7fa9437fe640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:37.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.754+0000 7fa9437fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:37.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.754+0000 7fa9437fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:37.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.754+0000 7fa9437fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:37.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.754+0000 7fa9437fe640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fa9480af260 0x7fa9480b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:37.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.754+0000 7fa9437fe640 1 -- 172.21.15.3:0/2366430713 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7fa94c0dc880 con 0x7fa95c074750 2024-05-21T17:41:37.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.754+0000 7fa95a7fc640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fa9480af260 0x7fa9480b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:37.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.755+0000 7fa95a7fc640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fa9480af260 0x7fa9480b1720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa95c13d1c0 tx=0x7fa950007cf0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:37.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:37.758+0000 7fa9437fe640 1 -- 172.21.15.3:0/2366430713 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa94c0a6630 con 0x7fa95c074750 2024-05-21T17:41:38.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.006+0000 7fa96142a640 1 -- 172.21.15.3:0/2366430713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa95c0a3150 con 0x7fa95c074750 2024-05-21T17:41:38.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.011+0000 7fa9437fe640 1 -- 172.21.15.3:0/2366430713 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+283558 (secure 0 0 0) 0x7fa94c0a5f30 con 0x7fa95c074750 2024-05-21T17:41:38.016 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:38.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.014+0000 7fa96142a640 10 mgrc shutdown 2024-05-21T17:41:38.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.014+0000 7fa96142a640 1 -- 172.21.15.3:0/2366430713 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fa9480af260 msgr2=0x7fa9480b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.014+0000 7fa96142a640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fa9480af260 0x7fa9480b1720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa95c13d1c0 tx=0x7fa950007cf0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.014+0000 7fa96142a640 1 -- 172.21.15.3:0/2366430713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c074750 msgr2=0x7fa95c13c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.014+0000 7fa96142a640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c074750 0x7fa95c13c050 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00c7f0 tx=0x7fa94c00cbf0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.015+0000 7fa96142a640 1 -- 172.21.15.3:0/2366430713 shutdown_connections 2024-05-21T17:41:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.015+0000 7fa96142a640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fa9480af260 0x7fa9480b1720 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.015+0000 7fa96142a640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa95c14c7f0 0x7fa95c0b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.015+0000 7fa96142a640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa95c0a1720 0x7fa95c13db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.015+0000 7fa96142a640 1 --2- 172.21.15.3:0/2366430713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa95c074750 0x7fa95c13c050 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.015+0000 7fa96142a640 1 -- 172.21.15.3:0/2366430713 >> 172.21.15.3:0/2366430713 conn(0x7fa95c06ccf0 msgr2=0x7fa95c1473d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.015+0000 7fa96142a640 1 -- 172.21.15.3:0/2366430713 shutdown_connections 2024-05-21T17:41:38.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.015+0000 7fa96142a640 1 -- 172.21.15.3:0/2366430713 wait complete. 2024-05-21T17:41:38.027 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"z","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":4157202067},{"type":"v1","addr":"172.21.15.203:6825","nonce":4157202067}]},"active_addr":"172.21.15.203:6825/4157202067","active_change":"2024-05-21T17:41:21.510830+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2972207335}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":515614797}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2295098024}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":1911408330}]}]} 2024-05-21T17:41:38.032 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-21T17:41:38.032 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-21T17:41:38.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T17:41:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.169+0000 7f59be6fc640 1 Processor -- start 2024-05-21T17:41:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.169+0000 7f59be6fc640 1 -- start start 2024-05-21T17:41:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59be6fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f59b80a5f10 0x7f59b80a1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59be6fc640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b814d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59be6fc640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b80a4400 con 0x7f59b80a93c0 2024-05-21T17:41:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59be6fc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f59b806e330 con 0x7f59b80a5f10 2024-05-21T17:41:38.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59be6fc640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f59b806e5d0 con 0x7f59b8073f00 2024-05-21T17:41:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59b7fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f59b80a93c0 0x7f59b806da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60682/0 (socket says 172.21.15.3:60682) 2024-05-21T17:41:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59bcc99640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b814d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59b7fff640 1 -- 172.21.15.3:0/2224411283 learned_addr learned my addr 172.21.15.3:0/2224411283 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59b77fe640 1 --2- 172.21.15.3:0/2224411283 >> v2:172.21.15.203:3300/0 conn(0x7f59b80a5f10 0x7f59b80a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59bcc99640 1 -- 172.21.15.3:0/2224411283 >> v2:172.21.15.203:3300/0 conn(0x7f59b80a5f10 msgr2=0x7f59b80a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59bcc99640 1 --2- 172.21.15.3:0/2224411283 >> v2:172.21.15.203:3300/0 conn(0x7f59b80a5f10 0x7f59b80a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59bcc99640 1 -- 172.21.15.3:0/2224411283 >> v1:172.21.15.3:6789/0 conn(0x7f59b80a93c0 legacy=0x7f59b806da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59b6ffd640 1 -- 172.21.15.3:0/2224411283 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1549035734 0 0) 0x7f59b80a4400 con 0x7f59b80a93c0 2024-05-21T17:41:38.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.170+0000 7f59bcc99640 1 -- 172.21.15.3:0/2224411283 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b80a97a0 con 0x7f59b8073f00 2024-05-21T17:41:38.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59bcc99640 1 --2- 172.21.15.3:0/2224411283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b814d680 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009d90 tx=0x7f59b002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9875681f263a9f89 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59b6ffd640 1 -- 172.21.15.3:0/2224411283 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0039070 con 0x7f59b8073f00 2024-05-21T17:41:38.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59b6ffd640 1 -- 172.21.15.3:0/2224411283 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b002cbc0 con 0x7f59b8073f00 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59b6ffd640 1 -- 172.21.15.3:0/2224411283 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0034440 con 0x7f59b8073f00 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59be6fc640 1 -- 172.21.15.3:0/2224411283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 msgr2=0x7f59b814d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59be6fc640 1 --2- 172.21.15.3:0/2224411283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b814d680 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009d90 tx=0x7f59b002be10 comp rx=0 tx=0).stop 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59be6fc640 1 -- 172.21.15.3:0/2224411283 shutdown_connections 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59be6fc640 1 --2- 172.21.15.3:0/2224411283 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b814d680 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59be6fc640 1 --2- 172.21.15.3:0/2224411283 >> v2:172.21.15.203:3300/0 conn(0x7f59b80a5f10 0x7f59b80a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59be6fc640 1 -- 172.21.15.3:0/2224411283 >> 172.21.15.3:0/2224411283 conn(0x7f59b806ccf0 msgr2=0x7f59b80a6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.171+0000 7f59be6fc640 1 -- 172.21.15.3:0/2224411283 shutdown_connections 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.172+0000 7f59be6fc640 1 -- 172.21.15.3:0/2224411283 wait complete. 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.172+0000 7f59be6fc640 1 Processor -- start 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59be6fc640 1 -- start start 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59be6fc640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b8139ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59be6fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59b80a5f10 0x7f59b8072e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59be6fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b80a93c0 0x7f59b809bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59be6fc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59b80744f0 con 0x7f59b80a5f10 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59be6fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8074370 con 0x7f59b80a93c0 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59be6fc640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f59b8074670 con 0x7f59b8073f00 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59b7fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b8139ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59b77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59b80a5f10 0x7f59b8072e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59b7fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b8139ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40884/0 (socket says 172.21.15.3:40884) 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59b77fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59b80a5f10 0x7f59b8072e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51420/0 (socket says 172.21.15.3:51420) 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.173+0000 7f59b7fff640 1 -- 172.21.15.3:0/3120901611 learned_addr learned my addr 172.21.15.3:0/3120901611 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59bcc99640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b80a93c0 0x7f59b809bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59b7fff640 1 -- 172.21.15.3:0/3120901611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b80a93c0 msgr2=0x7f59b809bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59b7fff640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b80a93c0 0x7f59b809bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59b7fff640 1 -- 172.21.15.3:0/3120901611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59b80a5f10 msgr2=0x7f59b8072e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59b7fff640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59b80a5f10 0x7f59b8072e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59b7fff640 1 -- 172.21.15.3:0/3120901611 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b8136920 con 0x7f59b8073f00 2024-05-21T17:41:38.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59b77fe640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59b80a5f10 0x7f59b8072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:38.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59b7fff640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b8139ee0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c430 tx=0x7f59a800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59b4ff9640 1 -- 172.21.15.3:0/3120901611 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8017070 con 0x7f59b8073f00 2024-05-21T17:41:38.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59b4ff9640 1 -- 172.21.15.3:0/3120901611 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a8005ce0 con 0x7f59b8073f00 2024-05-21T17:41:38.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59be6fc640 1 -- 172.21.15.3:0/3120901611 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b8146ff0 con 0x7f59b8073f00 2024-05-21T17:41:38.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59b4ff9640 1 -- 172.21.15.3:0/3120901611 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8007310 con 0x7f59b8073f00 2024-05-21T17:41:38.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.174+0000 7f59be6fc640 1 -- 172.21.15.3:0/3120901611 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b809b580 con 0x7f59b8073f00 2024-05-21T17:41:38.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.175+0000 7f59be6fc640 1 -- 172.21.15.3:0/3120901611 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b8069a90 con 0x7f59b8073f00 2024-05-21T17:41:38.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.176+0000 7f59b4ff9640 1 -- 172.21.15.3:0/3120901611 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f59a801a070 con 0x7f59b8073f00 2024-05-21T17:41:38.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.176+0000 7f59b4ff9640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:38.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.177+0000 7f59b4ff9640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:38.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.177+0000 7f59b4ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.177+0000 7f59b4ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.177+0000 7f59b4ff9640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f59a40af290 0x7f59a40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.177+0000 7f59b4ff9640 1 -- 172.21.15.3:0/3120901611 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f59a80dc040 con 0x7f59b8073f00 2024-05-21T17:41:38.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.177+0000 7f59b77fe640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f59a40af290 0x7f59a40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.178+0000 7f59b77fe640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f59a40af290 0x7f59a40b1750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f59b80a0d40 tx=0x7f59ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.180+0000 7f59b4ff9640 1 -- 172.21.15.3:0/3120901611 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f59a8021020 con 0x7f59b8073f00 2024-05-21T17:41:38.267 INFO:tasks.ceph.mds.c.smithi003.stderr:2024-05-21T17:41:38.266+0000 7fcc4aa1b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-21T17:41:38.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.376+0000 7f59be6fc640 1 -- 172.21.15.3:0/3120901611 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59b80a6630 con 0x7f59b8073f00 2024-05-21T17:41:38.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.377+0000 7f59b4ff9640 1 -- 172.21.15.3:0/3120901611 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+13521 (secure 0 0 0) 0x7f59a80ace20 con 0x7f59b8073f00 2024-05-21T17:41:38.378 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:38.378 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442","created":"2024-05-21T17:41:17.034337+0000","modified":"2024-05-21T17:41:34.184816+0000","last_up_change":"2024-05-21T17:41:25.136644+0000","last_in_change":"2024-05-21T17:41:18.179591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T17:41:25.537561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-21T17:41:26.016407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-05-21T17:41:30.560078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-05-21T17:41:31.496059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7ac7071-219d-4cf4-9f8a-977f495fd0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6801","nonce":3296405141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6803","nonce":3296405141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6807","nonce":3296405141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6805","nonce":3296405141}]},"public_addr":"172.21.15.3:6801/3296405141","cluster_addr":"172.21.15.3:6803/3296405141","heartbeat_back_addr":"172.21.15.3:6807/3296405141","heartbeat_front_addr":"172.21.15.3:6805/3296405141","state":["exists","up"]},{"osd":1,"uuid":"89df5105-5900-4ffa-b8c1-6f480865a085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6801","nonce":1777872358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6803","nonce":1777872358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6810","nonce":1777872358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6805","nonce":1777872358}]},"public_addr":"172.21.15.203:6801/1777872358","cluster_addr":"172.21.15.203:6803/1777872358","heartbeat_back_addr":"172.21.15.203:6810/1777872358","heartbeat_front_addr":"172.21.15.203:6805/1777872358","state":["exists","up"]},{"osd":2,"uuid":"15f83334-8efc-40c2-af0c-eeed45923086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6817","nonce":1069390381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6819","nonce":1069390381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6823","nonce":1069390381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6821","nonce":1069390381}]},"public_addr":"172.21.15.203:6817/1069390381","cluster_addr":"172.21.15.203:6819/1069390381","heartbeat_back_addr":"172.21.15.203:6823/1069390381","heartbeat_front_addr":"172.21.15.203:6821/1069390381","state":["exists","up"]},{"osd":3,"uuid":"6a27ac8e-f957-4455-b477-3090081766c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6808","nonce":1072938612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6811","nonce":1072938612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6815","nonce":1072938612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6813","nonce":1072938612}]},"public_addr":"172.21.15.203:6808/1072938612","cluster_addr":"172.21.15.203:6811/1072938612","heartbeat_back_addr":"172.21.15.203:6815/1072938612","heartbeat_front_addr":"172.21.15.203:6813/1072938612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.224040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.344461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.319792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.332782+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T17:41:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 10 mgrc shutdown 2024-05-21T17:41:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 1 -- 172.21.15.3:0/3120901611 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f59a40af290 msgr2=0x7f59a40b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f59a40af290 0x7f59a40b1750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f59b80a0d40 tx=0x7f59ac008040 comp rx=0 tx=0).stop 2024-05-21T17:41:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 1 -- 172.21.15.3:0/3120901611 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 msgr2=0x7f59b8139ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b8139ee0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c430 tx=0x7f59a800c8f0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 1 -- 172.21.15.3:0/3120901611 shutdown_connections 2024-05-21T17:41:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f59a40af290 0x7f59a40b1750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b80a93c0 0x7f59b809bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59b80a5f10 0x7f59b8072e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 1 --2- 172.21.15.3:0/3120901611 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f59b8073f00 0x7f59b8139ee0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.379+0000 7f59be6fc640 1 -- 172.21.15.3:0/3120901611 >> 172.21.15.3:0/3120901611 conn(0x7f59b806ccf0 msgr2=0x7f59b80a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.380+0000 7f59be6fc640 1 -- 172.21.15.3:0/3120901611 shutdown_connections 2024-05-21T17:41:38.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.380+0000 7f59be6fc640 1 -- 172.21.15.3:0/3120901611 wait complete. 2024-05-21T17:41:38.392 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-21T17:41:38.392 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T17:41:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.494+0000 7f4c597d7640 1 Processor -- start 2024-05-21T17:41:38.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.494+0000 7f4c597d7640 1 -- start start 2024-05-21T17:41:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c597d7640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c540ac330 0x7f4c540ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c597d7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4c54149720 0x7f4c54149b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c597d7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c54146b20 con 0x7f4c5414a040 2024-05-21T17:41:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c597d7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4c540a8530 con 0x7f4c54149720 2024-05-21T17:41:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c597d7640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f4c54146de0 con 0x7f4c540ac330 2024-05-21T17:41:38.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c52ffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c540ac330 0x7f4c540ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c52ffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c540ac330 0x7f4c540ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40888/0 (socket says 172.21.15.3:40888) 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c52ffd640 1 -- 172.21.15.3:0/2389721121 learned_addr learned my addr 172.21.15.3:0/2389721121 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c4bfff640 1 --2- 172.21.15.3:0/2389721121 >> v2:172.21.15.203:3300/0 conn(0x7f4c54149720 0x7f4c54149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c52ffd640 1 -- 172.21.15.3:0/2389721121 >> v2:172.21.15.203:3300/0 conn(0x7f4c54149720 msgr2=0x7f4c54149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c52ffd640 1 --2- 172.21.15.3:0/2389721121 >> v2:172.21.15.203:3300/0 conn(0x7f4c54149720 0x7f4c54149b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c52ffd640 1 -- 172.21.15.3:0/2389721121 >> v1:172.21.15.203:6789/0 conn(0x7f4c5414a040 legacy=0x7f4c54145640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.495+0000 7f4c52ffd640 1 -- 172.21.15.3:0/2389721121 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5414a4d0 con 0x7f4c540ac330 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.496+0000 7f4c52ffd640 1 --2- 172.21.15.3:0/2389721121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c540ac330 0x7f4c540ad480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009870 tx=0x7f4c4402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=297138e56ad890a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.496+0000 7f4c527fc640 1 -- 172.21.15.3:0/2389721121 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44035020 con 0x7f4c540ac330 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.496+0000 7f4c527fc640 1 -- 172.21.15.3:0/2389721121 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c44031640 con 0x7f4c540ac330 2024-05-21T17:41:38.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.496+0000 7f4c527fc640 1 -- 172.21.15.3:0/2389721121 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4402d040 con 0x7f4c540ac330 2024-05-21T17:41:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.497+0000 7f4c597d7640 1 -- 172.21.15.3:0/2389721121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c540ac330 msgr2=0x7f4c540ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.497+0000 7f4c597d7640 1 --2- 172.21.15.3:0/2389721121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c540ac330 0x7f4c540ad480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009870 tx=0x7f4c4402b8f0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.497+0000 7f4c597d7640 1 -- 172.21.15.3:0/2389721121 shutdown_connections 2024-05-21T17:41:38.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.497+0000 7f4c597d7640 1 --2- 172.21.15.3:0/2389721121 >> v2:172.21.15.203:3300/0 conn(0x7f4c54149720 0x7f4c54149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.497+0000 7f4c597d7640 1 --2- 172.21.15.3:0/2389721121 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c540ac330 0x7f4c540ad480 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.497+0000 7f4c597d7640 1 -- 172.21.15.3:0/2389721121 >> 172.21.15.3:0/2389721121 conn(0x7f4c540aab00 msgr2=0x7f4c540aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.497+0000 7f4c597d7640 1 -- 172.21.15.3:0/2389721121 shutdown_connections 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.497+0000 7f4c597d7640 1 -- 172.21.15.3:0/2389721121 wait complete. 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c597d7640 1 Processor -- start 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c597d7640 1 -- start start 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c597d7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c540ac330 0x7f4c541400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c597d7640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c54149720 0x7f4c54137760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c597d7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c5414a040 0x7f4c5413c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c597d7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5406ccf0 con 0x7f4c540ac330 2024-05-21T17:41:38.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c597d7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4c540ac830 con 0x7f4c5414a040 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c597d7640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5406ce70 con 0x7f4c54149720 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c4bfff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c54149720 0x7f4c54137760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c52ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c540ac330 0x7f4c541400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c4bfff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c54149720 0x7f4c54137760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40890/0 (socket says 172.21.15.3:40890) 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.498+0000 7f4c4bfff640 1 -- 172.21.15.3:0/2924745700 learned_addr learned my addr 172.21.15.3:0/2924745700 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c4bfff640 1 -- 172.21.15.3:0/2924745700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c5414a040 msgr2=0x7f4c5413c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c4bfff640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c5414a040 0x7f4c5413c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c4bfff640 1 -- 172.21.15.3:0/2924745700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c540ac330 msgr2=0x7f4c541400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c4bfff640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c540ac330 0x7f4c541400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c4bfff640 1 -- 172.21.15.3:0/2924745700 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c540a7cd0 con 0x7f4c54149720 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c4bfff640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c54149720 0x7f4c54137760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00d830 tx=0x7f4c3c00dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c4b7fe640 1 -- 172.21.15.3:0/2924745700 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00b6c0 con 0x7f4c54149720 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c4b7fe640 1 -- 172.21.15.3:0/2924745700 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c3c019070 con 0x7f4c54149720 2024-05-21T17:41:38.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c597d7640 1 -- 172.21.15.3:0/2924745700 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c54138020 con 0x7f4c54149720 2024-05-21T17:41:38.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c4b7fe640 1 -- 172.21.15.3:0/2924745700 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00ba80 con 0x7f4c54149720 2024-05-21T17:41:38.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.499+0000 7f4c597d7640 1 -- 172.21.15.3:0/2924745700 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c541385a0 con 0x7f4c54149720 2024-05-21T17:41:38.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.500+0000 7f4c597d7640 1 -- 172.21.15.3:0/2924745700 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c54069350 con 0x7f4c54149720 2024-05-21T17:41:38.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.501+0000 7f4c4b7fe640 1 -- 172.21.15.3:0/2924745700 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f4c3c00bc20 con 0x7f4c54149720 2024-05-21T17:41:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.501+0000 7f4c4b7fe640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.502+0000 7f4c4b7fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.502+0000 7f4c4b7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.502+0000 7f4c4b7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.502+0000 7f4c4b7fe640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c340af260 0x7f4c340b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.502+0000 7f4c4b7fe640 1 -- 172.21.15.3:0/2924745700 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f4c3c0dce60 con 0x7f4c54149720 2024-05-21T17:41:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.502+0000 7f4c52ffd640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c340af260 0x7f4c340b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.503+0000 7f4c52ffd640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c340af260 0x7f4c340b1720 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4c44036000 tx=0x7f4c4403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.506+0000 7f4c4b7fe640 1 -- 172.21.15.3:0/2924745700 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4c3c0a6b70 con 0x7f4c54149720 2024-05-21T17:41:38.697 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.696+0000 7f4c597d7640 1 -- 172.21.15.3:0/2924745700 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c54069a90 con 0x7f4c54149720 2024-05-21T17:41:38.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.697+0000 7f4c4b7fe640 1 -- 172.21.15.3:0/2924745700 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+13521 (secure 0 0 0) 0x7f4c3c0a6470 con 0x7f4c54149720 2024-05-21T17:41:38.699 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:38.699 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442","created":"2024-05-21T17:41:17.034337+0000","modified":"2024-05-21T17:41:34.184816+0000","last_up_change":"2024-05-21T17:41:25.136644+0000","last_in_change":"2024-05-21T17:41:18.179591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T17:41:25.537561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-21T17:41:26.016407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-05-21T17:41:30.560078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-05-21T17:41:31.496059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7ac7071-219d-4cf4-9f8a-977f495fd0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6801","nonce":3296405141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6803","nonce":3296405141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6807","nonce":3296405141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6805","nonce":3296405141}]},"public_addr":"172.21.15.3:6801/3296405141","cluster_addr":"172.21.15.3:6803/3296405141","heartbeat_back_addr":"172.21.15.3:6807/3296405141","heartbeat_front_addr":"172.21.15.3:6805/3296405141","state":["exists","up"]},{"osd":1,"uuid":"89df5105-5900-4ffa-b8c1-6f480865a085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6801","nonce":1777872358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6803","nonce":1777872358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6810","nonce":1777872358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6805","nonce":1777872358}]},"public_addr":"172.21.15.203:6801/1777872358","cluster_addr":"172.21.15.203:6803/1777872358","heartbeat_back_addr":"172.21.15.203:6810/1777872358","heartbeat_front_addr":"172.21.15.203:6805/1777872358","state":["exists","up"]},{"osd":2,"uuid":"15f83334-8efc-40c2-af0c-eeed45923086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6817","nonce":1069390381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6819","nonce":1069390381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6823","nonce":1069390381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6821","nonce":1069390381}]},"public_addr":"172.21.15.203:6817/1069390381","cluster_addr":"172.21.15.203:6819/1069390381","heartbeat_back_addr":"172.21.15.203:6823/1069390381","heartbeat_front_addr":"172.21.15.203:6821/1069390381","state":["exists","up"]},{"osd":3,"uuid":"6a27ac8e-f957-4455-b477-3090081766c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6808","nonce":1072938612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6811","nonce":1072938612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6815","nonce":1072938612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6813","nonce":1072938612}]},"public_addr":"172.21.15.203:6808/1072938612","cluster_addr":"172.21.15.203:6811/1072938612","heartbeat_back_addr":"172.21.15.203:6815/1072938612","heartbeat_front_addr":"172.21.15.203:6813/1072938612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.224040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.344461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.319792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.332782+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T17:41:38.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 10 mgrc shutdown 2024-05-21T17:41:38.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 1 -- 172.21.15.3:0/2924745700 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c340af260 msgr2=0x7f4c340b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c340af260 0x7f4c340b1720 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4c44036000 tx=0x7f4c4403c040 comp rx=0 tx=0).stop 2024-05-21T17:41:38.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 1 -- 172.21.15.3:0/2924745700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c54149720 msgr2=0x7f4c54137760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c54149720 0x7f4c54137760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00d830 tx=0x7f4c3c00dcf0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 1 -- 172.21.15.3:0/2924745700 shutdown_connections 2024-05-21T17:41:38.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c340af260 0x7f4c340b1720 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c5414a040 0x7f4c5413c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c54149720 0x7f4c54137760 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 1 --2- 172.21.15.3:0/2924745700 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c540ac330 0x7f4c541400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.699+0000 7f4c597d7640 1 -- 172.21.15.3:0/2924745700 >> 172.21.15.3:0/2924745700 conn(0x7f4c540aab00 msgr2=0x7f4c54145ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.700+0000 7f4c597d7640 1 -- 172.21.15.3:0/2924745700 shutdown_connections 2024-05-21T17:41:38.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.700+0000 7f4c597d7640 1 -- 172.21.15.3:0/2924745700 wait complete. 2024-05-21T17:41:38.712 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-21T17:41:38.713 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-21T17:41:38.713 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-21T17:41:38.714 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-21T17:41:38.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.821+0000 7f5b53c45640 1 Processor -- start 2024-05-21T17:41:38.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f317580d640 1 Processor -- start 2024-05-21T17:41:38.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f13af033640 1 Processor -- start 2024-05-21T17:41:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f5945d7e640 1 Processor -- start 2024-05-21T17:41:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f13af033640 1 -- start start 2024-05-21T17:41:38.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f5945d7e640 1 -- start start 2024-05-21T17:41:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f317580d640 1 -- start start 2024-05-21T17:41:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f5b53c45640 1 -- start start 2024-05-21T17:41:38.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f13af033640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13a81387f0 0x7f13a80ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f13af033640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80ad6d0 0x7f13a8150b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f13af033640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a8138c80 con 0x7f13a80abf70 2024-05-21T17:41:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f317580d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3170074370 0x7f31701339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f13af033640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f13a8139810 con 0x7f13a81387f0 2024-05-21T17:41:38.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f13af033640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f13a8138f20 con 0x7f13a80ad6d0 2024-05-21T17:41:38.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f5945d7e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f59400a86d0 0x7f5940149df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f317580d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f317013a660 0x7f317014e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.822+0000 7f5b53c45640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b4c0a8ae0 0x7f5b4c1460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b53c45640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5b4c0a7e40 0x7f5b4c146610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f13ad5d0640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80ad6d0 0x7f13a8150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f316effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3170074370 0x7f31701339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f317580d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31700a9a10 con 0x7f31700ab040 2024-05-21T17:41:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f317580d640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3170074750 con 0x7f3170074370 2024-05-21T17:41:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5945d7e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594014a330 0x7f5940150b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5945d7e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f594014a740 con 0x7f59400a6710 2024-05-21T17:41:38.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f13ad5d0640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80ad6d0 0x7f13a8150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40904/0 (socket says 172.21.15.3:40904) 2024-05-21T17:41:38.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f13ad5d0640 1 -- 172.21.15.3:0/1122446652 learned_addr learned my addr 172.21.15.3:0/1122446652 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:38.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b53c45640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b4c06e3b0 con 0x7f5b4c0a1350 2024-05-21T17:41:38.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5945d7e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f594006fa60 con 0x7f59400a86d0 2024-05-21T17:41:38.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5945d7e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f59401499a0 con 0x7f594014a330 2024-05-21T17:41:38.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f593ffff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594014a330 0x7f5940150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f593ffff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594014a330 0x7f5940150b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40924/0 (socket says 172.21.15.3:40924) 2024-05-21T17:41:38.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f593ffff640 1 -- 172.21.15.3:0/1097322541 learned_addr learned my addr 172.21.15.3:0/1097322541 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:38.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f317580d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f317013a330 con 0x7f317013a660 2024-05-21T17:41:38.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b53c45640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f5b4c0a6430 con 0x7f5b4c0a7e40 2024-05-21T17:41:38.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b53c45640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4c0a1900 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b521e2640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f5b4c0a1350 0x7f5b4c0a4840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60694/0 (socket says 172.21.15.3:60694) 2024-05-21T17:41:38.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f316effd640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3170074370 0x7f31701339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51434/0 (socket says 172.21.15.3:51434) 2024-05-21T17:41:38.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b521e2640 1 -- 172.21.15.3:0/4229209095 learned_addr learned my addr 172.21.15.3:0/4229209095 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:38.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f316effd640 1 -- 172.21.15.3:0/1939328033 learned_addr learned my addr 172.21.15.3:0/1939328033 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:38.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f316e7fc640 1 --1- 172.21.15.3:0/1939328033 >> v1:172.21.15.203:6789/0 conn(0x7f31700ab040 0x7f31700ad680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.3:40950/0 (socket says 172.21.15.3:40950) 2024-05-21T17:41:38.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f13a7fff640 1 --2- 172.21.15.3:0/1122446652 >> v2:172.21.15.203:3300/0 conn(0x7f13a81387f0 0x7f13a80ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b511e0640 1 --2- 172.21.15.3:0/4229209095 >> v2:172.21.15.203:3300/0 conn(0x7f5b4c0a7e40 0x7f5b4c146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.833 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f593effd640 1 --2- 172.21.15.3:0/1097322541 >> v2:172.21.15.3:3300/0 conn(0x7f59400a86d0 0x7f5940149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b519e1640 1 --2- 172.21.15.3:0/4229209095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b4c0a8ae0 0x7f5b4c1460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f13ad5d0640 1 -- 172.21.15.3:0/1122446652 >> v2:172.21.15.203:3300/0 conn(0x7f13a81387f0 msgr2=0x7f13a80ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:38.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f13ad5d0640 1 --2- 172.21.15.3:0/1122446652 >> v2:172.21.15.203:3300/0 conn(0x7f13a81387f0 0x7f13a80ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f593ffff640 1 -- 172.21.15.3:0/1097322541 >> v2:172.21.15.3:3300/0 conn(0x7f59400a86d0 msgr2=0x7f5940149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:38.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f13ad5d0640 1 -- 172.21.15.3:0/1122446652 >> v1:172.21.15.203:6789/0 conn(0x7f13a80abf70 legacy=0x7f13a806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f13ad5d0640 1 -- 172.21.15.3:0/1122446652 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a813c970 con 0x7f13a80ad6d0 2024-05-21T17:41:38.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f593ffff640 1 --2- 172.21.15.3:0/1097322541 >> v2:172.21.15.3:3300/0 conn(0x7f59400a86d0 0x7f5940149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b519e1640 1 -- 172.21.15.3:0/4229209095 >> v2:172.21.15.203:3300/0 conn(0x7f5b4c0a7e40 msgr2=0x7f5b4c146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b519e1640 1 --2- 172.21.15.3:0/4229209095 >> v2:172.21.15.203:3300/0 conn(0x7f5b4c0a7e40 0x7f5b4c146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b519e1640 1 -- 172.21.15.3:0/4229209095 >> v1:172.21.15.3:6789/0 conn(0x7f5b4c0a1350 legacy=0x7f5b4c0a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f5b519e1640 1 -- 172.21.15.3:0/4229209095 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c0a5df0 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.823+0000 7f593ffff640 1 -- 172.21.15.3:0/1097322541 >> v1:172.21.15.203:6789/0 conn(0x7f59400a6710 legacy=0x7f59400a8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f593ffff640 1 -- 172.21.15.3:0/1097322541 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59400a59a0 con 0x7f594014a330 2024-05-21T17:41:38.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f316effd640 1 -- 172.21.15.3:0/1939328033 >> v2:172.21.15.203:3300/0 conn(0x7f317013a660 msgr2=0x7f317014e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:38.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f5b511e0640 1 --2- 172.21.15.3:0/4229209095 >> v2:172.21.15.203:3300/0 conn(0x7f5b4c0a7e40 0x7f5b4c146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:38.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f593effd640 1 --2- 172.21.15.3:0/1097322541 >> v2:172.21.15.3:3300/0 conn(0x7f59400a86d0 0x7f5940149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:38.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f316f7fe640 1 --2- 172.21.15.3:0/1939328033 >> v2:172.21.15.203:3300/0 conn(0x7f317013a660 0x7f317014e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f316dffb640 1 -- 172.21.15.3:0/1939328033 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2632194405 0 0) 0x7f31700a9a10 con 0x7f31700ab040 2024-05-21T17:41:38.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f316effd640 1 --2- 172.21.15.3:0/1939328033 >> v2:172.21.15.203:3300/0 conn(0x7f317013a660 0x7f317014e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f316effd640 1 -- 172.21.15.3:0/1939328033 >> v1:172.21.15.203:6789/0 conn(0x7f31700ab040 legacy=0x7f31700ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f316effd640 1 -- 172.21.15.3:0/1939328033 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3170136660 con 0x7f3170074370 2024-05-21T17:41:38.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f316f7fe640 1 --2- 172.21.15.3:0/1939328033 >> v2:172.21.15.203:3300/0 conn(0x7f317013a660 0x7f317014e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:38.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f13ad5d0640 1 --2- 172.21.15.3:0/1122446652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80ad6d0 0x7f13a8150b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009e50 tx=0x7f13a002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a745eee72379fc8b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.839 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f5b519e1640 1 --2- 172.21.15.3:0/4229209095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b4c0a8ae0 0x7f5b4c1460d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800ba40 tx=0x7f5b4802dff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5c71710cc64c6a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.824+0000 7f13a77fe640 1 -- 172.21.15.3:0/1122446652 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a002ca70 con 0x7f13a80ad6d0 2024-05-21T17:41:38.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f13a77fe640 1 -- 172.21.15.3:0/1122446652 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a002cc10 con 0x7f13a80ad6d0 2024-05-21T17:41:38.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f13a77fe640 1 -- 172.21.15.3:0/1122446652 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a00343f0 con 0x7f13a80ad6d0 2024-05-21T17:41:38.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f5b509df640 1 -- 172.21.15.3:0/4229209095 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4803b070 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f5b509df640 1 -- 172.21.15.3:0/4229209095 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4802eda0 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.840 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f5b509df640 1 -- 172.21.15.3:0/4229209095 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48036660 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f593ffff640 1 --2- 172.21.15.3:0/1097322541 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594014a330 0x7f5940150b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5938009e50 tx=0x7f593802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c4a18ea185f5f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f316effd640 1 --2- 172.21.15.3:0/1939328033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3170074370 0x7f31701339f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f317013a4b0 tx=0x7f316002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759a591f7a3aa27b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.841 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f593e7fc640 1 -- 172.21.15.3:0/1097322541 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593802ca70 con 0x7f594014a330 2024-05-21T17:41:38.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f593e7fc640 1 -- 172.21.15.3:0/1097322541 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593802cc10 con 0x7f594014a330 2024-05-21T17:41:38.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f593e7fc640 1 -- 172.21.15.3:0/1097322541 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59380343f0 con 0x7f594014a330 2024-05-21T17:41:38.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f316dffb640 1 -- 172.21.15.3:0/1939328033 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316002ca70 con 0x7f3170074370 2024-05-21T17:41:38.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f316dffb640 1 -- 172.21.15.3:0/1939328033 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316002cc10 con 0x7f3170074370 2024-05-21T17:41:38.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f5b53c45640 1 -- 172.21.15.3:0/4229209095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b4c0a8ae0 msgr2=0x7f5b4c1460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f13af033640 1 -- 172.21.15.3:0/1122446652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80ad6d0 msgr2=0x7f13a8150b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f5b53c45640 1 --2- 172.21.15.3:0/4229209095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b4c0a8ae0 0x7f5b4c1460d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800ba40 tx=0x7f5b4802dff0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f13af033640 1 --2- 172.21.15.3:0/1122446652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80ad6d0 0x7f13a8150b20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009e50 tx=0x7f13a002bed0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f316dffb640 1 -- 172.21.15.3:0/1939328033 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316002ca70 con 0x7f3170074370 2024-05-21T17:41:38.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.825+0000 7f5b53c45640 1 -- 172.21.15.3:0/4229209095 shutdown_connections 2024-05-21T17:41:38.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5b53c45640 1 --2- 172.21.15.3:0/4229209095 >> v2:172.21.15.203:3300/0 conn(0x7f5b4c0a7e40 0x7f5b4c146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.844 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5b53c45640 1 --2- 172.21.15.3:0/4229209095 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b4c0a8ae0 0x7f5b4c1460d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5b53c45640 1 -- 172.21.15.3:0/4229209095 >> 172.21.15.3:0/4229209095 conn(0x7f5b4c06ccf0 msgr2=0x7f5b4c0a6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f13af033640 1 -- 172.21.15.3:0/1122446652 shutdown_connections 2024-05-21T17:41:38.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f13af033640 1 --2- 172.21.15.3:0/1122446652 >> v2:172.21.15.203:3300/0 conn(0x7f13a81387f0 0x7f13a80ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f13af033640 1 --2- 172.21.15.3:0/1122446652 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80ad6d0 0x7f13a8150b20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f13af033640 1 -- 172.21.15.3:0/1122446652 >> 172.21.15.3:0/1122446652 conn(0x7f13a80aab70 msgr2=0x7f13a80aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f13af033640 1 -- 172.21.15.3:0/1122446652 shutdown_connections 2024-05-21T17:41:38.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f13af033640 1 -- 172.21.15.3:0/1122446652 wait complete. 2024-05-21T17:41:38.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5b53c45640 1 -- 172.21.15.3:0/4229209095 shutdown_connections 2024-05-21T17:41:38.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5b53c45640 1 -- 172.21.15.3:0/4229209095 wait complete. 2024-05-21T17:41:38.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5945d7e640 1 -- 172.21.15.3:0/1097322541 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594014a330 msgr2=0x7f5940150b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5945d7e640 1 --2- 172.21.15.3:0/1097322541 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594014a330 0x7f5940150b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5938009e50 tx=0x7f593802bed0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5945d7e640 1 -- 172.21.15.3:0/1097322541 shutdown_connections 2024-05-21T17:41:38.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5945d7e640 1 --2- 172.21.15.3:0/1097322541 >> v2:172.21.15.3:3300/0 conn(0x7f59400a86d0 0x7f5940149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f317580d640 1 -- 172.21.15.3:0/1939328033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3170074370 msgr2=0x7f31701339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5945d7e640 1 --2- 172.21.15.3:0/1097322541 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594014a330 0x7f5940150b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5945d7e640 1 -- 172.21.15.3:0/1097322541 >> 172.21.15.3:0/1097322541 conn(0x7f594006db20 msgr2=0x7f59400a7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f317580d640 1 --2- 172.21.15.3:0/1939328033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3170074370 0x7f31701339f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f317013a4b0 tx=0x7f316002bed0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.826+0000 7f5945d7e640 1 -- 172.21.15.3:0/1097322541 shutdown_connections 2024-05-21T17:41:38.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f317580d640 1 -- 172.21.15.3:0/1939328033 shutdown_connections 2024-05-21T17:41:38.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f317580d640 1 --2- 172.21.15.3:0/1939328033 >> v2:172.21.15.203:3300/0 conn(0x7f317013a660 0x7f317014e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f317580d640 1 --2- 172.21.15.3:0/1939328033 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3170074370 0x7f31701339f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f317580d640 1 -- 172.21.15.3:0/1939328033 >> 172.21.15.3:0/1939328033 conn(0x7f31700713c0 msgr2=0x7f317006d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f5945d7e640 1 -- 172.21.15.3:0/1097322541 wait complete. 2024-05-21T17:41:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f317580d640 1 -- 172.21.15.3:0/1939328033 shutdown_connections 2024-05-21T17:41:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f317580d640 1 -- 172.21.15.3:0/1939328033 wait complete. 2024-05-21T17:41:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f13af033640 1 Processor -- start 2024-05-21T17:41:38.850 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f13af033640 1 -- start start 2024-05-21T17:41:38.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f5b53c45640 1 Processor -- start 2024-05-21T17:41:38.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.827+0000 7f5b53c45640 1 -- start start 2024-05-21T17:41:38.851 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13af033640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80abf70 0x7f13a80a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13af033640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a80ad6d0 0x7f13a8147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f317580d640 1 Processor -- start 2024-05-21T17:41:38.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13acdcf640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80abf70 0x7f13a80a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13acdcf640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80abf70 0x7f13a80a7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40936/0 (socket says 172.21.15.3:40936) 2024-05-21T17:41:38.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13af033640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13a8148690 0x7f13a8148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13acdcf640 1 -- 172.21.15.3:0/3334444745 learned_addr learned my addr 172.21.15.3:0/3334444745 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:38.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f13a80adae0 con 0x7f13a8148690 2024-05-21T17:41:38.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f13a80a52e0 con 0x7f13a80ad6d0 2024-05-21T17:41:38.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13ad5d0640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13a8148690 0x7f13a8148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f13a80adc60 con 0x7f13a80abf70 2024-05-21T17:41:38.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f317580d640 1 -- start start 2024-05-21T17:41:38.854 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f5945d7e640 1 Processor -- start 2024-05-21T17:41:38.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13a7fff640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a80ad6d0 0x7f13a8147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13acdcf640 1 -- 172.21.15.3:0/3334444745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a80ad6d0 msgr2=0x7f13a8147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13acdcf640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a80ad6d0 0x7f13a8147620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13acdcf640 1 -- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13a8148690 msgr2=0x7f13a8148b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13acdcf640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13a8148690 0x7f13a8148b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13acdcf640 1 -- 172.21.15.3:0/3334444745 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a81df110 con 0x7f13a80abf70 2024-05-21T17:41:38.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f13ad5d0640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f13a8148690 0x7f13a8148b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:38.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f5945d7e640 1 -- start start 2024-05-21T17:41:38.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f13acdcf640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80abf70 0x7f13a80a7b40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f139c00c400 tx=0x7f139c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f5b53c45640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b4c0a1350 0x7f5b4c09b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.828+0000 7f5b53c45640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c0a8ae0 0x7f5b4c09abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b519e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b4c0a1350 0x7f5b4c09b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b511e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c0a8ae0 0x7f5b4c09abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b519e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b4c0a1350 0x7f5b4c09b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.3:38854/0 (socket says 172.21.15.3:38854) 2024-05-21T17:41:38.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b511e0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c0a8ae0 0x7f5b4c09abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51462/0 (socket says 172.21.15.3:51462) 2024-05-21T17:41:38.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b53c45640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b4c09cc20 0x7f5b4c1da7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c0a8630 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b519e1640 1 -- 172.21.15.3:0/1124336666 learned_addr learned my addr 172.21.15.3:0/1124336666 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:38.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c0a84b0 con 0x7f5b4c0a1350 2024-05-21T17:41:38.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4c146b50 con 0x7f5b4c09cc20 2024-05-21T17:41:38.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f13a57fa640 1 -- 172.21.15.3:0/3334444745 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c017070 con 0x7f13a80abf70 2024-05-21T17:41:38.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f317580d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3170074370 0x7f31700b6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f13a57fa640 1 -- 172.21.15.3:0/3334444745 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c0052d0 con 0x7f13a80abf70 2024-05-21T17:41:38.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a81df3b0 con 0x7f13a80abf70 2024-05-21T17:41:38.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f317580d640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31700ab040 0x7f31700b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f13a57fa640 1 -- 172.21.15.3:0/3334444745 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c003830 con 0x7f13a80abf70 2024-05-21T17:41:38.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a8147c90 con 0x7f13a80abf70 2024-05-21T17:41:38.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f317580d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317013a660 0x7f31700b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f317580d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f31700add90 con 0x7f3170074370 2024-05-21T17:41:38.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f317580d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f317013a040 con 0x7f317013a660 2024-05-21T17:41:38.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f317580d640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f31700adf10 con 0x7f31700ab040 2024-05-21T17:41:38.860 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f316effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3170074370 0x7f31700b6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f316effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3170074370 0x7f31700b6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51464/0 (socket says 172.21.15.3:51464) 2024-05-21T17:41:38.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f316effd640 1 -- 172.21.15.3:0/4043164539 learned_addr learned my addr 172.21.15.3:0/4043164539 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:38.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5945d7e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400a6710 0x7f5940145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f316f7fe640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317013a660 0x7f31700b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f316effd640 1 -- 172.21.15.3:0/4043164539 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31700ab040 msgr2=0x7f31700b8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5945d7e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59400a86d0 0x7f59400a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5945d7e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594014a330 0x7f5940142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5945d7e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f59400a6300 con 0x7f59400a86d0 2024-05-21T17:41:38.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5945d7e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f59400a6180 con 0x7f59400a6710 2024-05-21T17:41:38.862 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5945d7e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f59400700b0 con 0x7f594014a330 2024-05-21T17:41:38.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b511e0640 1 -- 172.21.15.3:0/1124336666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b4c09cc20 msgr2=0x7f5b4c1da7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b511e0640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f5b4c09cc20 0x7f5b4c1da7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.829+0000 7f5b511e0640 1 -- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b4c0a1350 msgr2=0x7f5b4c09b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5b511e0640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b4c0a1350 0x7f5b4c09b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5b511e0640 1 -- 172.21.15.3:0/1124336666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c09d520 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5b511e0640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c0a8ae0 0x7f5b4c09abf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00eb90 tx=0x7f5b3c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.863 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f593f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400a6710 0x7f5940145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f593f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400a6710 0x7f5940145070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.3:38872/0 (socket says 172.21.15.3:38872) 2024-05-21T17:41:38.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f593f7fe640 1 -- 172.21.15.3:0/558196446 learned_addr learned my addr 172.21.15.3:0/558196446 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:38.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f316e7fc640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31700ab040 0x7f31700b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f316effd640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31700ab040 0x7f31700b8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f316effd640 1 -- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317013a660 msgr2=0x7f31700b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.864 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f316effd640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317013a660 0x7f31700b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f316effd640 1 -- 172.21.15.3:0/4043164539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31701d9530 con 0x7f3170074370 2024-05-21T17:41:38.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f316e7fc640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f31700ab040 0x7f31700b8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:38.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f316f7fe640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f317013a660 0x7f31700b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T17:41:38.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5b427fc640 1 -- 172.21.15.3:0/1124336666 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c017070 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.865 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f316effd640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3170074370 0x7f31700b6b90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f316002c4e0 tx=0x7f3160004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5b427fc640 1 -- 172.21.15.3:0/1124336666 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c00eda0 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5b427fc640 1 -- 172.21.15.3:0/1124336666 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00ce70 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13a80b5c80 con 0x7f13a80abf70 2024-05-21T17:41:38.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f593effd640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59400a86d0 0x7f59400a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.830+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c09c440 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c1d71f0 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f3157fff640 1 -- 172.21.15.3:0/4043164539 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160050070 con 0x7f3170074370 2024-05-21T17:41:38.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f593f7fe640 1 -- 172.21.15.3:0/558196446 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594014a330 msgr2=0x7f5940142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f593f7fe640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f594014a330 0x7f5940142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f593f7fe640 1 -- 172.21.15.3:0/558196446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59400a86d0 msgr2=0x7f59400a2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f593f7fe640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f59400a86d0 0x7f59400a2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f593f7fe640 1 -- 172.21.15.3:0/558196446 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59400a3320 con 0x7f59400a6710 2024-05-21T17:41:38.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31701d97f0 con 0x7f3170074370 2024-05-21T17:41:38.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31700b85f0 con 0x7f3170074370 2024-05-21T17:41:38.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f3157fff640 1 -- 172.21.15.3:0/4043164539 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3160003b90 con 0x7f3170074370 2024-05-21T17:41:38.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.832+0000 7f3157fff640 1 -- 172.21.15.3:0/4043164539 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160004040 con 0x7f3170074370 2024-05-21T17:41:38.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.831+0000 7f593f7fe640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400a6710 0x7f5940145070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f593400c450 tx=0x7f593400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.832+0000 7f5927fff640 1 -- 172.21.15.3:0/558196446 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59340052d0 con 0x7f59400a6710 2024-05-21T17:41:38.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.832+0000 7f5927fff640 1 -- 172.21.15.3:0/558196446 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5934005470 con 0x7f59400a6710 2024-05-21T17:41:38.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.832+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b4c069a90 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.832+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3170069290 con 0x7f3170074370 2024-05-21T17:41:38.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.832+0000 7f5927fff640 1 -- 172.21.15.3:0/558196446 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934012850 con 0x7f59400a6710 2024-05-21T17:41:38.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.832+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59400a0fd0 con 0x7f59400a6710 2024-05-21T17:41:38.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.832+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59400a37b0 con 0x7f59400a6710 2024-05-21T17:41:38.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.833+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5940069290 con 0x7f59400a6710 2024-05-21T17:41:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.833+0000 7f13a57fa640 1 -- 172.21.15.3:0/3334444745 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f139c01a070 con 0x7f13a80abf70 2024-05-21T17:41:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.833+0000 7f13a57fa640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.833+0000 7f13a57fa640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.833+0000 7f13a57fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.833+0000 7f13a57fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.833+0000 7f13a57fa640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f138c0af3b0 0x7f138c0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.834+0000 7f13a57fa640 1 -- 172.21.15.3:0/3334444745 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f139c0dc0e0 con 0x7f13a80abf70 2024-05-21T17:41:38.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.834+0000 7f13a7fff640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f138c0af3b0 0x7f138c0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.834+0000 7f13a57fa640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:6800/3296405141,v1:172.21.15.3:6801/3296405141] conn(0x7f138c0b6f10 0x7f138c0b9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.834+0000 7f13a57fa640 1 -- 172.21.15.3:0/3334444745 --> [v2:172.21.15.3:6800/3296405141,v1:172.21.15.3:6801/3296405141] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f139c009e50 con 0x7f138c0b6f10 2024-05-21T17:41:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.834+0000 7f13a57fa640 1 -- 172.21.15.3:0/3334444745 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f139c0dc4d0 con 0x7f13a80abf70 2024-05-21T17:41:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.834+0000 7f13ad5d0640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:6800/3296405141,v1:172.21.15.3:6801/3296405141] conn(0x7f138c0b6f10 0x7f138c0b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.834+0000 7f13a7fff640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f138c0af3b0 0x7f138c0b1870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f13a80a82e0 tx=0x7f1394009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.834+0000 7f5b427fc640 1 -- 172.21.15.3:0/1124336666 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f5b3c01a070 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.834+0000 7f5b427fc640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.834+0000 7f13ad5d0640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:6800/3296405141,v1:172.21.15.3:6801/3296405141] conn(0x7f138c0b6f10 0x7f138c0b9350 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f5b427fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f5b427fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f5b427fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f5b427fc640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f5b300af2b0 0x7f5b300b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f3157fff640 1 -- 172.21.15.3:0/4043164539 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f3160005d10 con 0x7f3170074370 2024-05-21T17:41:38.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f3157fff640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:38.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f13a57fa640 1 -- 172.21.15.3:0/3334444745 <== osd.0 v2:172.21.15.3:6800/3296405141 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f139c009e50 con 0x7f138c0b6f10 2024-05-21T17:41:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f5b427fc640 1 -- 172.21.15.3:0/1124336666 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f5b3c0dcb60 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f5b519e1640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f5b300af2b0 0x7f5b300b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f5b427fc640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:6816/1069390381,v1:172.21.15.203:6817/1069390381] conn(0x7f5b300b6e10 0x7f5b300b9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f5b427fc640 1 -- 172.21.15.3:0/1124336666 --> [v2:172.21.15.203:6816/1069390381,v1:172.21.15.203:6817/1069390381] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b3c016ec0 con 0x7f5b300b6e10 2024-05-21T17:41:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.835+0000 7f5b427fc640 1 -- 172.21.15.3:0/1124336666 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f5b3c0dcff0 con 0x7f5b4c0a8ae0 2024-05-21T17:41:38.876 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f5b521e2640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:6816/1069390381,v1:172.21.15.203:6817/1069390381] conn(0x7f5b300b6e10 0x7f5b300b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f3157fff640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:38.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f3157fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f3157fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f3157fff640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f315c0af2e0 0x7f315c0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f5b519e1640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f5b300af2b0 0x7f5b300b1770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800b800 tx=0x7f5b48032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f3157fff640 1 -- 172.21.15.3:0/4043164539 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f3160107140 con 0x7f3170074370 2024-05-21T17:41:38.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f5b521e2640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:6816/1069390381,v1:172.21.15.203:6817/1069390381] conn(0x7f5b300b6e10 0x7f5b300b9250 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f5927fff640 1 -- 172.21.15.3:0/558196446 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f59340129f0 con 0x7f59400a6710 2024-05-21T17:41:38.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f5927fff640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:38.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f3157fff640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:6800/1777872358,v1:172.21.15.203:6801/1777872358] conn(0x7f315c0b6e40 0x7f315c0b9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f3157fff640 1 -- 172.21.15.3:0/4043164539 --> [v2:172.21.15.203:6800/1777872358,v1:172.21.15.203:6801/1777872358] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3160042e50 con 0x7f315c0b6e40 2024-05-21T17:41:38.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f3157fff640 1 -- 172.21.15.3:0/4043164539 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f3160107530 con 0x7f3170074370 2024-05-21T17:41:38.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.836+0000 7f316f7fe640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:6800/1777872358,v1:172.21.15.203:6801/1777872358] conn(0x7f315c0b6e40 0x7f315c0b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.837+0000 7f5927fff640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:38.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.837+0000 7f5927fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.837+0000 7f5927fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:38.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.837+0000 7f5927fff640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f592c0af330 0x7f592c0b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.837+0000 7f316f7fe640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:6800/1777872358,v1:172.21.15.203:6801/1777872358] conn(0x7f315c0b6e40 0x7f315c0b9280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.837+0000 7f5b427fc640 1 -- 172.21.15.3:0/1124336666 <== osd.2 v2:172.21.15.203:6816/1069390381 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b3c016ec0 con 0x7f5b300b6e10 2024-05-21T17:41:38.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.837+0000 7f5927fff640 1 -- 172.21.15.3:0/558196446 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f59340dce50 con 0x7f59400a6710 2024-05-21T17:41:38.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.837+0000 7f5927fff640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:6806/1072938612,v1:172.21.15.203:6808/1072938612] conn(0x7f592c0b6ea0 0x7f592c0b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:38.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.837+0000 7f5927fff640 1 -- 172.21.15.3:0/558196446 --> [v2:172.21.15.203:6806/1072938612,v1:172.21.15.203:6808/1072938612] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f592c0b99b0 con 0x7f592c0b6ea0 2024-05-21T17:41:38.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.837+0000 7f5927fff640 1 -- 172.21.15.3:0/558196446 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f59340dd200 con 0x7f59400a6710 2024-05-21T17:41:38.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.838+0000 7f593ffff640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:6806/1072938612,v1:172.21.15.203:6808/1072938612] conn(0x7f592c0b6ea0 0x7f592c0b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.882 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.838+0000 7f3157fff640 1 -- 172.21.15.3:0/4043164539 <== osd.1 v2:172.21.15.203:6800/1777872358 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3160042e50 con 0x7f315c0b6e40 2024-05-21T17:41:38.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.838+0000 7f593ffff640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:6806/1072938612,v1:172.21.15.203:6808/1072938612] conn(0x7f592c0b6ea0 0x7f592c0b92e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.839+0000 7f5927fff640 1 -- 172.21.15.3:0/558196446 <== osd.3 v2:172.21.15.203:6806/1072938612 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f592c0b99b0 con 0x7f592c0b6ea0 2024-05-21T17:41:38.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.852+0000 7f316e7fc640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f315c0af2e0 0x7f315c0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.852+0000 7f593effd640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f592c0af330 0x7f592c0b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:38.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.852+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 --> [v2:172.21.15.203:6816/1069390381,v1:172.21.15.203:6817/1069390381] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b4c0a6050 con 0x7f5b300b6e10 2024-05-21T17:41:38.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.852+0000 7f5b427fc640 1 -- 172.21.15.3:0/1124336666 <== osd.2 v2:172.21.15.203:6816/1069390381 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5b4c0a6050 con 0x7f5b300b6e10 2024-05-21T17:41:38.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.852+0000 7f316e7fc640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f315c0af2e0 0x7f315c0b17a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3164000f60 tx=0x7f3164009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.852+0000 7f593effd640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f592c0af330 0x7f592c0b17f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f594009fec0 tx=0x7f5930009110 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:38.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.852+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:6816/1069390381,v1:172.21.15.203:6817/1069390381] conn(0x7f5b300b6e10 msgr2=0x7f5b300b9250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.852+0000 7f5b53c45640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:6816/1069390381,v1:172.21.15.203:6817/1069390381] conn(0x7f5b300b6e10 0x7f5b300b9250 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f5b53c45640 10 mgrc shutdown 2024-05-21T17:41:38.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f5b300af2b0 msgr2=0x7f5b300b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f5b53c45640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f5b300af2b0 0x7f5b300b1770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800b800 tx=0x7f5b48032000 comp rx=0 tx=0).stop 2024-05-21T17:41:38.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c0a8ae0 msgr2=0x7f5b4c09abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f5b53c45640 1 --2- 172.21.15.3:0/1124336666 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5b4c0a8ae0 0x7f5b4c09abf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00eb90 tx=0x7f5b3c00c520 comp rx=0 tx=0).stop 2024-05-21T17:41:38.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f5b521e2640 1 -- 172.21.15.3:0/1124336666 reap_dead start 2024-05-21T17:41:38.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 shutdown_connections 2024-05-21T17:41:38.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 >> 172.21.15.3:0/1124336666 conn(0x7f5b4c06ccf0 msgr2=0x7f5b4c0a6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 shutdown_connections 2024-05-21T17:41:38.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f5b53c45640 1 -- 172.21.15.3:0/1124336666 wait complete. 2024-05-21T17:41:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 --> [v2:172.21.15.203:6800/1777872358,v1:172.21.15.203:6801/1777872358] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f31700b7d80 con 0x7f315c0b6e40 2024-05-21T17:41:38.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.853+0000 7f3157fff640 1 -- 172.21.15.3:0/4043164539 <== osd.1 v2:172.21.15.203:6800/1777872358 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f31700b7d80 con 0x7f315c0b6e40 2024-05-21T17:41:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:6800/1777872358,v1:172.21.15.203:6801/1777872358] conn(0x7f315c0b6e40 msgr2=0x7f315c0b9280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:6800/1777872358,v1:172.21.15.203:6801/1777872358] conn(0x7f315c0b6e40 0x7f315c0b9280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 10 mgrc shutdown 2024-05-21T17:41:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f315c0af2e0 msgr2=0x7f315c0b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f315c0af2e0 0x7f315c0b17a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3164000f60 tx=0x7f3164009040 comp rx=0 tx=0).stop 2024-05-21T17:41:38.888 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3170074370 msgr2=0x7f31700b6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 1 --2- 172.21.15.3:0/4043164539 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3170074370 0x7f31700b6b90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f316002c4e0 tx=0x7f3160004a40 comp rx=0 tx=0).stop 2024-05-21T17:41:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f316f7fe640 1 -- 172.21.15.3:0/4043164539 reap_dead start 2024-05-21T17:41:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 shutdown_connections 2024-05-21T17:41:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 >> 172.21.15.3:0/4043164539 conn(0x7f31700713c0 msgr2=0x7f317006c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 shutdown_connections 2024-05-21T17:41:38.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.854+0000 7f317580d640 1 -- 172.21.15.3:0/4043164539 wait complete. 2024-05-21T17:41:38.890 INFO:teuthology.orchestra.run.smithi003.stdout:38654705670 2024-05-21T17:41:38.890 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-21T17:41:38.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.865+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 --> [v2:172.21.15.3:6800/3296405141,v1:172.21.15.3:6801/3296405141] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13a80b5c80 con 0x7f138c0b6f10 2024-05-21T17:41:38.891 INFO:teuthology.orchestra.run.smithi003.stdout:38654705670 2024-05-21T17:41:38.892 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-21T17:41:38.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.865+0000 7f13a57fa640 1 -- 172.21.15.3:0/3334444745 <== osd.0 v2:172.21.15.3:6800/3296405141 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f13a002c870 con 0x7f138c0b6f10 2024-05-21T17:41:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.865+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:6800/3296405141,v1:172.21.15.3:6801/3296405141] conn(0x7f138c0b6f10 msgr2=0x7f138c0b9350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.865+0000 7f13af033640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:6800/3296405141,v1:172.21.15.3:6801/3296405141] conn(0x7f138c0b6f10 0x7f138c0b9350 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.865+0000 7f13af033640 10 mgrc shutdown 2024-05-21T17:41:38.893 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.866+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f138c0af3b0 msgr2=0x7f138c0b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.866+0000 7f13af033640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f138c0af3b0 0x7f138c0b1870 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f13a80a82e0 tx=0x7f1394009190 comp rx=0 tx=0).stop 2024-05-21T17:41:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.866+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80abf70 msgr2=0x7f13a80a7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.866+0000 7f13af033640 1 --2- 172.21.15.3:0/3334444745 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f13a80abf70 0x7f13a80a7b40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f139c00c400 tx=0x7f139c00c8c0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.866+0000 7f13ad5d0640 1 -- 172.21.15.3:0/3334444745 reap_dead start 2024-05-21T17:41:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.866+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 shutdown_connections 2024-05-21T17:41:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.866+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 >> 172.21.15.3:0/3334444745 conn(0x7f13a80aab70 msgr2=0x7f13a80a8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.866+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 shutdown_connections 2024-05-21T17:41:38.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.866+0000 7f13af033640 1 -- 172.21.15.3:0/3334444745 wait complete. 2024-05-21T17:41:38.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.868+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 --> [v2:172.21.15.203:6806/1072938612,v1:172.21.15.203:6808/1072938612] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f594006f2e0 con 0x7f592c0b6ea0 2024-05-21T17:41:38.895 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5927fff640 1 -- 172.21.15.3:0/558196446 <== osd.3 v2:172.21.15.203:6806/1072938612 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f594006f2e0 con 0x7f592c0b6ea0 2024-05-21T17:41:38.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:6806/1072938612,v1:172.21.15.203:6808/1072938612] conn(0x7f592c0b6ea0 msgr2=0x7f592c0b92e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:6806/1072938612,v1:172.21.15.203:6808/1072938612] conn(0x7f592c0b6ea0 0x7f592c0b92e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:38.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 10 mgrc shutdown 2024-05-21T17:41:38.896 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f592c0af330 msgr2=0x7f592c0b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f592c0af330 0x7f592c0b17f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f594009fec0 tx=0x7f5930009110 comp rx=0 tx=0).stop 2024-05-21T17:41:38.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400a6710 msgr2=0x7f5940145070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:38.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 1 --2- 172.21.15.3:0/558196446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59400a6710 0x7f5940145070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f593400c450 tx=0x7f593400c910 comp rx=0 tx=0).stop 2024-05-21T17:41:38.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f593ffff640 1 -- 172.21.15.3:0/558196446 reap_dead start 2024-05-21T17:41:38.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 shutdown_connections 2024-05-21T17:41:38.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 >> 172.21.15.3:0/558196446 conn(0x7f594006db20 msgr2=0x7f59400b6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:38.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 shutdown_connections 2024-05-21T17:41:38.897 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:38.869+0000 7f5945d7e640 1 -- 172.21.15.3:0/558196446 wait complete. 2024-05-21T17:41:38.898 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2024-05-21T17:41:38.898 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-21T17:41:38.899 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2024-05-21T17:41:38.899 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-21T17:41:39.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.040+0000 7fccffc21640 1 Processor -- start 2024-05-21T17:41:39.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.040+0000 7fb467fde640 1 Processor -- start 2024-05-21T17:41:39.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.040+0000 7fbd2b942640 1 Processor -- start 2024-05-21T17:41:39.042 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.040+0000 7f4c71705640 1 Processor -- start 2024-05-21T17:41:39.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.040+0000 7f4c71705640 1 -- start start 2024-05-21T17:41:39.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.040+0000 7fb467fde640 1 -- start start 2024-05-21T17:41:39.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.040+0000 7fccffc21640 1 -- start start 2024-05-21T17:41:39.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.040+0000 7fbd2b942640 1 -- start start 2024-05-21T17:41:39.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccffc21640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 0x7fccf80ac150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccffc21640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccf814a5c0 0x7fccf80a9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccffc21640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fccf80a9880 0x7fccf814aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccffc21640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fccf806d730 con 0x7fccf814a5c0 2024-05-21T17:41:39.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccffc21640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fccf806c880 con 0x7fccf80a9880 2024-05-21T17:41:39.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccffc21640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fccf806cb20 con 0x7fccf806d350 2024-05-21T17:41:39.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fbd2b942640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbd240b60b0 0x7fbd24145730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fbd2b942640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd24145c70 0x7fbd24150a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fbd2b942640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd24073c90 con 0x7fbd240728f0 2024-05-21T17:41:39.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fbd2b942640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbd24072480 con 0x7fbd240b60b0 2024-05-21T17:41:39.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fbd2b942640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fbd24073a40 con 0x7fbd24145c70 2024-05-21T17:41:39.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccfd1bc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccf814a5c0 0x7fccf80a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccfd1bc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fccf814a5c0 0x7fccf80a9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51496/0 (socket says 172.21.15.3:51496) 2024-05-21T17:41:39.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccfd1bc640 1 -- 172.21.15.3:0/3093538286 learned_addr learned my addr 172.21.15.3:0/3093538286 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:39.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fb467fde640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb4600b8e50 0x7fb4601447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccfe1be640 1 --2- 172.21.15.3:0/3093538286 >> v2:172.21.15.203:3300/0 conn(0x7fccf80a9880 0x7fccf814aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fb467fde640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600a5a70 0x7fb4600a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7f4c71705640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4c6c075540 0x7f4c6c0aa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fbd29edf640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd24145c70 0x7fbd24150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7f4c71705640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c145800 0x7f4c6c145c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fccfd9bd640 1 --2- 172.21.15.3:0/3093538286 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 0x7fccf80ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c71705640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c6c0a9c30 con 0x7f4c6c0a94c0 2024-05-21T17:41:39.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fbd29edf640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd24145c70 0x7fbd24150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40958/0 (socket says 172.21.15.3:40958) 2024-05-21T17:41:39.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.041+0000 7fbd29edf640 1 -- 172.21.15.3:0/1344006567 learned_addr learned my addr 172.21.15.3:0/1344006567 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fb467fde640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fb46006c410 0x7fb46006c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c71705640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4c6c075920 con 0x7f4c6c075540 2024-05-21T17:41:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c71705640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6c0a9ef0 con 0x7f4c6c145800 2024-05-21T17:41:39.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c6a7fc640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4c6c0a94c0 0x7f4c6c0a6c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60696/0 (socket says 172.21.15.3:60696) 2024-05-21T17:41:39.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fb465d7a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb4600b8e50 0x7fb4601447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fb465d7a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb4600b8e50 0x7fb4601447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.3:38878/0 (socket says 172.21.15.3:38878) 2024-05-21T17:41:39.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c6a7fc640 1 -- 172.21.15.3:0/2289902750 learned_addr learned my addr 172.21.15.3:0/2289902750 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:39.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c6b7fe640 1 --2- 172.21.15.3:0/2289902750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c145800 0x7f4c6c145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c6affd640 1 --2- 172.21.15.3:0/2289902750 >> v2:172.21.15.203:3300/0 conn(0x7f4c6c075540 0x7f4c6c0aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fb465d7a640 1 -- 172.21.15.3:0/1170949768 learned_addr learned my addr 172.21.15.3:0/1170949768 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:39.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fb465579640 1 --2- 172.21.15.3:0/1170949768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600a5a70 0x7fb4600a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fccfe1be640 1 -- 172.21.15.3:0/3093538286 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 msgr2=0x7fccf80ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fccfe1be640 1 --2- 172.21.15.3:0/3093538286 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 0x7fccf80ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fccfe1be640 1 -- 172.21.15.3:0/3093538286 >> v2:172.21.15.3:3300/0 conn(0x7fccf814a5c0 msgr2=0x7fccf80a9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fccfe1be640 1 --2- 172.21.15.3:0/3093538286 >> v2:172.21.15.3:3300/0 conn(0x7fccf814a5c0 0x7fccf80a9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.053 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fccfe1be640 1 -- 172.21.15.3:0/3093538286 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf80ae330 con 0x7fccf80a9880 2024-05-21T17:41:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fb467fde640 1 -- 172.21.15.3:0/1170949768 --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fb4600b9230 con 0x7fb46006c410 2024-05-21T17:41:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fb467fde640 1 -- 172.21.15.3:0/1170949768 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb4600aafd0 con 0x7fb4600b8e50 2024-05-21T17:41:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fb467fde640 1 -- 172.21.15.3:0/1170949768 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fb4600a7330 con 0x7fb4600a5a70 2024-05-21T17:41:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fb46657b640 1 --2- 172.21.15.3:0/1170949768 >> v2:172.21.15.3:3300/0 conn(0x7fb46006c410 0x7fb46006c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c6b7fe640 1 -- 172.21.15.3:0/2289902750 >> v2:172.21.15.203:3300/0 conn(0x7f4c6c075540 msgr2=0x7f4c6c0aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c6b7fe640 1 --2- 172.21.15.3:0/2289902750 >> v2:172.21.15.203:3300/0 conn(0x7f4c6c075540 0x7f4c6c0aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c6b7fe640 1 -- 172.21.15.3:0/2289902750 >> v1:172.21.15.3:6789/0 conn(0x7f4c6c0a94c0 legacy=0x7f4c6c0a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7f4c6b7fe640 1 -- 172.21.15.3:0/2289902750 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6c0a55a0 con 0x7f4c6c145800 2024-05-21T17:41:39.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fbd1bfff640 1 -- 172.21.15.3:0/1344006567 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238417405 0 0) 0x7fbd24073c90 con 0x7fbd240728f0 2024-05-21T17:41:39.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fbd1bfff640 1 -- 172.21.15.3:0/1344006567 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd10003520 con 0x7fbd240728f0 2024-05-21T17:41:39.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fbd28edd640 1 --2- 172.21.15.3:0/1344006567 >> v2:172.21.15.3:3300/0 conn(0x7fbd240b60b0 0x7fbd24145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fbd29edf640 1 -- 172.21.15.3:0/1344006567 >> v2:172.21.15.3:3300/0 conn(0x7fbd240b60b0 msgr2=0x7fbd24145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:39.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fbd29edf640 1 --2- 172.21.15.3:0/1344006567 >> v2:172.21.15.3:3300/0 conn(0x7fbd240b60b0 0x7fbd24145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fbd29edf640 1 -- 172.21.15.3:0/1344006567 >> v1:172.21.15.203:6789/0 conn(0x7fbd240728f0 legacy=0x7fbd2406c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.042+0000 7fbd29edf640 1 -- 172.21.15.3:0/1344006567 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd24073f50 con 0x7fbd24145c70 2024-05-21T17:41:39.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7fb46657b640 1 -- 172.21.15.3:0/1170949768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600a5a70 msgr2=0x7fb4600a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:39.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7fb46657b640 1 --2- 172.21.15.3:0/1170949768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600a5a70 0x7fb4600a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7fb46657b640 1 -- 172.21.15.3:0/1170949768 >> v2:172.21.15.203:3300/0 conn(0x7fb4600b8e50 msgr2=0x7fb4601447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7fb46657b640 1 --2- 172.21.15.3:0/1170949768 >> v2:172.21.15.203:3300/0 conn(0x7fb4600b8e50 0x7fb4601447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7fb46657b640 1 -- 172.21.15.3:0/1170949768 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4600a75f0 con 0x7fb46006c410 2024-05-21T17:41:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7fb465d7a640 1 --2- 172.21.15.3:0/1170949768 >> v2:172.21.15.203:3300/0 conn(0x7fb4600b8e50 0x7fb4601447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7f4c6b7fe640 1 --2- 172.21.15.3:0/2289902750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c145800 0x7f4c6c145c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0aa070 tx=0x7f4c6402be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bfbea213fa2999b server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.057 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7fbd28edd640 1 --2- 172.21.15.3:0/1344006567 >> v2:172.21.15.3:3300/0 conn(0x7fbd240b60b0 0x7fbd24145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T17:41:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7fccfe1be640 1 --2- 172.21.15.3:0/3093538286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf80a9880 0x7fccf814aca0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fccf80acb00 tx=0x7fccf002bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ce91393a066d147 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7f4c69ffb640 1 -- 172.21.15.3:0/2289902750 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64039070 con 0x7f4c6c145800 2024-05-21T17:41:39.058 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.043+0000 7fb46657b640 1 --2- 172.21.15.3:0/1170949768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb46006c410 0x7fb46006c880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb46006f1a0 tx=0x7fb458030690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ae53c4b14414dd server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7f4c69ffb640 1 -- 172.21.15.3:0/2289902750 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c6402cbb0 con 0x7f4c6c145800 2024-05-21T17:41:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7f4c69ffb640 1 -- 172.21.15.3:0/2289902750 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c64034440 con 0x7f4c6c145800 2024-05-21T17:41:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fccfc9bb640 1 -- 172.21.15.3:0/3093538286 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0039070 con 0x7fccf80a9880 2024-05-21T17:41:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fccfc9bb640 1 -- 172.21.15.3:0/3093538286 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf002cab0 con 0x7fccf80a9880 2024-05-21T17:41:39.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fccfc9bb640 1 -- 172.21.15.3:0/3093538286 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0034440 con 0x7fccf80a9880 2024-05-21T17:41:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fbd29edf640 1 --2- 172.21.15.3:0/1344006567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd24145c70 0x7fbd24150a40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009e50 tx=0x7fbd1c02bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d14d9097351603 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fb464d78640 1 -- 172.21.15.3:0/1170949768 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458035710 con 0x7fb46006c410 2024-05-21T17:41:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fb464d78640 1 -- 172.21.15.3:0/1170949768 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4580358b0 con 0x7fb46006c410 2024-05-21T17:41:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fb464d78640 1 -- 172.21.15.3:0/1170949768 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458035c80 con 0x7fb46006c410 2024-05-21T17:41:39.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7f4c71705640 1 -- 172.21.15.3:0/2289902750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c145800 msgr2=0x7f4c6c145c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7f4c71705640 1 --2- 172.21.15.3:0/2289902750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c145800 0x7f4c6c145c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0aa070 tx=0x7f4c6402be00 comp rx=0 tx=0).stop 2024-05-21T17:41:39.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fbd1bfff640 1 -- 172.21.15.3:0/1344006567 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c02ca80 con 0x7fbd24145c70 2024-05-21T17:41:39.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fccffc21640 1 -- 172.21.15.3:0/3093538286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf80a9880 msgr2=0x7fccf814aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fccffc21640 1 --2- 172.21.15.3:0/3093538286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf80a9880 0x7fccf814aca0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fccf80acb00 tx=0x7fccf002bd00 comp rx=0 tx=0).stop 2024-05-21T17:41:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7f4c71705640 1 -- 172.21.15.3:0/2289902750 shutdown_connections 2024-05-21T17:41:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7f4c71705640 1 --2- 172.21.15.3:0/2289902750 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c145800 0x7f4c6c145c50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fbd1bfff640 1 -- 172.21.15.3:0/1344006567 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c02cc20 con 0x7fbd24145c70 2024-05-21T17:41:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fbd1bfff640 1 -- 172.21.15.3:0/1344006567 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0343f0 con 0x7fbd24145c70 2024-05-21T17:41:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7f4c71705640 1 --2- 172.21.15.3:0/2289902750 >> v2:172.21.15.203:3300/0 conn(0x7f4c6c075540 0x7f4c6c0aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7f4c71705640 1 -- 172.21.15.3:0/2289902750 >> 172.21.15.3:0/2289902750 conn(0x7f4c6c0a74a0 msgr2=0x7f4c6c0a78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.044+0000 7fb467fde640 1 -- 172.21.15.3:0/1170949768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb46006c410 msgr2=0x7fb46006c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fb467fde640 1 --2- 172.21.15.3:0/1170949768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb46006c410 0x7fb46006c880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb46006f1a0 tx=0x7fb458030690 comp rx=0 tx=0).stop 2024-05-21T17:41:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7f4c71705640 1 -- 172.21.15.3:0/2289902750 shutdown_connections 2024-05-21T17:41:39.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fb467fde640 1 -- 172.21.15.3:0/1170949768 shutdown_connections 2024-05-21T17:41:39.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fccffc21640 1 -- 172.21.15.3:0/3093538286 shutdown_connections 2024-05-21T17:41:39.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fb467fde640 1 --2- 172.21.15.3:0/1170949768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb46006c410 0x7fb46006c880 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fb467fde640 1 --2- 172.21.15.3:0/1170949768 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600a5a70 0x7fb4600a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fb467fde640 1 --2- 172.21.15.3:0/1170949768 >> v2:172.21.15.203:3300/0 conn(0x7fb4600b8e50 0x7fb4601447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fb467fde640 1 -- 172.21.15.3:0/1170949768 >> 172.21.15.3:0/1170949768 conn(0x7fb46006d700 msgr2=0x7fb46006db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:39.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fccffc21640 1 --2- 172.21.15.3:0/3093538286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf80a9880 0x7fccf814aca0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fccffc21640 1 --2- 172.21.15.3:0/3093538286 >> v2:172.21.15.3:3300/0 conn(0x7fccf814a5c0 0x7fccf80a9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fccffc21640 1 --2- 172.21.15.3:0/3093538286 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 0x7fccf80ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.065 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fccffc21640 1 -- 172.21.15.3:0/3093538286 >> 172.21.15.3:0/3093538286 conn(0x7fccf80bbc20 msgr2=0x7fccf80bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fb467fde640 1 -- 172.21.15.3:0/1170949768 shutdown_connections 2024-05-21T17:41:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fb467fde640 1 -- 172.21.15.3:0/1170949768 wait complete. 2024-05-21T17:41:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7f4c71705640 1 -- 172.21.15.3:0/2289902750 wait complete. 2024-05-21T17:41:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fccffc21640 1 -- 172.21.15.3:0/3093538286 shutdown_connections 2024-05-21T17:41:39.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fccffc21640 1 -- 172.21.15.3:0/3093538286 wait complete. 2024-05-21T17:41:39.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fbd2b942640 1 -- 172.21.15.3:0/1344006567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd24145c70 msgr2=0x7fbd24150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fbd2b942640 1 --2- 172.21.15.3:0/1344006567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd24145c70 0x7fbd24150a40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009e50 tx=0x7fbd1c02bee0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.045+0000 7fbd2b942640 1 -- 172.21.15.3:0/1344006567 shutdown_connections 2024-05-21T17:41:39.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.046+0000 7fbd2b942640 1 --2- 172.21.15.3:0/1344006567 >> v2:172.21.15.3:3300/0 conn(0x7fbd240b60b0 0x7fbd24145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.046+0000 7fbd2b942640 1 --2- 172.21.15.3:0/1344006567 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd24145c70 0x7fbd24150a40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.046+0000 7fbd2b942640 1 -- 172.21.15.3:0/1344006567 >> 172.21.15.3:0/1344006567 conn(0x7fbd240713c0 msgr2=0x7fbd2406d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:39.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.046+0000 7fbd2b942640 1 -- 172.21.15.3:0/1344006567 shutdown_connections 2024-05-21T17:41:39.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.046+0000 7fbd2b942640 1 -- 172.21.15.3:0/1344006567 wait complete. 2024-05-21T17:41:39.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.046+0000 7f4c71705640 1 Processor -- start 2024-05-21T17:41:39.068 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.046+0000 7f4c71705640 1 -- start start 2024-05-21T17:41:39.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.046+0000 7fb467fde640 1 Processor -- start 2024-05-21T17:41:39.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.046+0000 7fb467fde640 1 -- start start 2024-05-21T17:41:39.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c71705640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c075540 0x7f4c6c073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c71705640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c0a94c0 0x7f4c6c0b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c71705640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c6c145800 0x7f4c6c0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c71705640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c150a30 con 0x7f4c6c145800 2024-05-21T17:41:39.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c71705640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c0a9910 con 0x7f4c6c0a94c0 2024-05-21T17:41:39.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c71705640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f4c6c150bb0 con 0x7f4c6c075540 2024-05-21T17:41:39.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6affd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c075540 0x7f4c6c073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6affd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c075540 0x7f4c6c073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40980/0 (socket says 172.21.15.3:40980) 2024-05-21T17:41:39.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fccffc21640 1 Processor -- start 2024-05-21T17:41:39.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6affd640 1 -- 172.21.15.3:0/3597917067 learned_addr learned my addr 172.21.15.3:0/3597917067 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:39.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6b7fe640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c6c145800 0x7f4c6c0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fccffc21640 1 -- start start 2024-05-21T17:41:39.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb467fde640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb46006c410 0x7fb46013e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb467fde640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4600a5a70 0x7fb4600ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6a7fc640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c0a94c0 0x7f4c6c0b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb467fde640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600b8e50 0x7fb4601457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb467fde640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb46006f690 con 0x7fb4600a5a70 2024-05-21T17:41:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb467fde640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb4600a6fd0 con 0x7fb46006c410 2024-05-21T17:41:39.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb467fde640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fb460070100 con 0x7fb4600b8e50 2024-05-21T17:41:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb46657b640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600b8e50 0x7fb4601457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb46657b640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600b8e50 0x7fb4601457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40988/0 (socket says 172.21.15.3:40988) 2024-05-21T17:41:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb46657b640 1 -- 172.21.15.3:0/837002327 learned_addr learned my addr 172.21.15.3:0/837002327 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb465d7a640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb46006c410 0x7fb46013e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6affd640 1 -- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c0a94c0 msgr2=0x7f4c6c0b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6affd640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c0a94c0 0x7f4c6c0b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6affd640 1 -- 172.21.15.3:0/3597917067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c6c145800 msgr2=0x7f4c6c0b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6affd640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c6c145800 0x7f4c6c0b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6affd640 1 -- 172.21.15.3:0/3597917067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6c13e910 con 0x7f4c6c075540 2024-05-21T17:41:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6b7fe640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c6c145800 0x7f4c6c0b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6a7fc640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c0a94c0 0x7f4c6c0b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:39.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7fb465579640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4600a5a70 0x7fb4600ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.047+0000 7f4c6affd640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c075540 0x7f4c6c073f70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00e9b0 tx=0x7f4c5c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7f4c537fe640 1 -- 172.21.15.3:0/3597917067 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c017070 con 0x7f4c6c075540 2024-05-21T17:41:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7f4c537fe640 1 -- 172.21.15.3:0/3597917067 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c5c00c9a0 con 0x7f4c6c075540 2024-05-21T17:41:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7f4c537fe640 1 -- 172.21.15.3:0/3597917067 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00cd90 con 0x7f4c6c075540 2024-05-21T17:41:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7f4c71705640 1 -- 172.21.15.3:0/3597917067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6c13ebd0 con 0x7f4c6c075540 2024-05-21T17:41:39.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fbd2b942640 1 Processor -- start 2024-05-21T17:41:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fb46657b640 1 -- 172.21.15.3:0/837002327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb46006c410 msgr2=0x7fb46013e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fb46657b640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb46006c410 0x7fb46013e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fb46657b640 1 -- 172.21.15.3:0/837002327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4600a5a70 msgr2=0x7fb4600ac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fb46657b640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4600a5a70 0x7fb4600ac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fb46657b640 1 -- 172.21.15.3:0/837002327 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4600ad360 con 0x7fb4600b8e50 2024-05-21T17:41:39.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fb465579640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4600a5a70 0x7fb4600ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7f4c71705640 1 -- 172.21.15.3:0/3597917067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c6c13de60 con 0x7f4c6c075540 2024-05-21T17:41:39.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fbd2b942640 1 -- start start 2024-05-21T17:41:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fccffc21640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 0x7fccf81d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fccffc21640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf80a9880 0x7fccf806f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fccffc21640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf814a5c0 0x7fccf80a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fccfd9bd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 0x7fccf81d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fccfd9bd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 0x7fccf81d59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40994/0 (socket says 172.21.15.3:40994) 2024-05-21T17:41:39.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fccfd9bd640 1 -- 172.21.15.3:0/3826963854 learned_addr learned my addr 172.21.15.3:0/3826963854 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:39.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fb46657b640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600b8e50 0x7fb4601457c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb458030ca0 tx=0x7fb458004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.048+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fccf814b570 con 0x7fccf80a9880 2024-05-21T17:41:39.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fccfd1bc640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf80a9880 0x7fccf806f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fccfe1be640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf814a5c0 0x7fccf80a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fccf814b3f0 con 0x7fccf814a5c0 2024-05-21T17:41:39.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fccf814dc20 con 0x7fccf806d350 2024-05-21T17:41:39.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fccfd9bd640 1 -- 172.21.15.3:0/3826963854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf814a5c0 msgr2=0x7fccf80a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fccfd9bd640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf814a5c0 0x7fccf80a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fccfd9bd640 1 -- 172.21.15.3:0/3826963854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf80a9880 msgr2=0x7fccf806f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fccfd9bd640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf80a9880 0x7fccf806f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fccfd9bd640 1 -- 172.21.15.3:0/3826963854 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf80a7270 con 0x7fccf806d350 2024-05-21T17:41:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fb4567fc640 1 -- 172.21.15.3:0/837002327 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458037b40 con 0x7fb4600b8e50 2024-05-21T17:41:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fb4567fc640 1 -- 172.21.15.3:0/837002327 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb458037ce0 con 0x7fb4600b8e50 2024-05-21T17:41:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fb467fde640 1 -- 172.21.15.3:0/837002327 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb460145370 con 0x7fb4600b8e50 2024-05-21T17:41:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fb4567fc640 1 -- 172.21.15.3:0/837002327 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458042660 con 0x7fb4600b8e50 2024-05-21T17:41:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fb467fde640 1 -- 172.21.15.3:0/837002327 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46013ebd0 con 0x7fb4600b8e50 2024-05-21T17:41:39.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fbd2b942640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd240728f0 0x7fbd240b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fccfd1bc640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf80a9880 0x7fccf806f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fbd2b942640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd240b60b0 0x7fbd240b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fbd2b942640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd24145c70 0x7fbd240b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.049+0000 7fbd296de640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd240728f0 0x7fbd240b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fbd2b942640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24073260 con 0x7fbd240b60b0 2024-05-21T17:41:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fbd29edf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd24145c70 0x7fbd240b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fbd29edf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd24145c70 0x7fbd240b8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.3:38928/0 (socket says 172.21.15.3:38928) 2024-05-21T17:41:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fbd29edf640 1 -- 172.21.15.3:0/1623917862 learned_addr learned my addr 172.21.15.3:0/1623917862 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fbd28edd640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd240b60b0 0x7fbd240b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24071ff0 con 0x7fbd24145c70 2024-05-21T17:41:39.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fbd24074910 con 0x7fbd240728f0 2024-05-21T17:41:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fccfd9bd640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 0x7fccf81d59d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fccf400d830 tx=0x7fccf400dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7f4c71705640 1 -- 172.21.15.3:0/3597917067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c6c069290 con 0x7f4c6c075540 2024-05-21T17:41:39.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fccee7fc640 1 -- 172.21.15.3:0/3826963854 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf400b6c0 con 0x7fccf806d350 2024-05-21T17:41:39.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fccee7fc640 1 -- 172.21.15.3:0/3826963854 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf4019070 con 0x7fccf806d350 2024-05-21T17:41:39.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fccee7fc640 1 -- 172.21.15.3:0/3826963854 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf400ba80 con 0x7fccf806d350 2024-05-21T17:41:39.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf8070ec0 con 0x7fccf806d350 2024-05-21T17:41:39.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf80a6680 con 0x7fccf806d350 2024-05-21T17:41:39.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.050+0000 7fbd29edf640 1 -- 172.21.15.3:0/1623917862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd240728f0 msgr2=0x7fbd240b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.051+0000 7fbd29edf640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd240728f0 0x7fbd240b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.051+0000 7fbd29edf640 1 -- 172.21.15.3:0/1623917862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd240b60b0 msgr2=0x7fbd240b7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.051+0000 7fbd29edf640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd240b60b0 0x7fbd240b7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.051+0000 7fbd29edf640 1 -- 172.21.15.3:0/1623917862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd240b8580 con 0x7fbd24145c70 2024-05-21T17:41:39.086 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.051+0000 7fb467fde640 1 -- 172.21.15.3:0/837002327 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4601423f0 con 0x7fb4600b8e50 2024-05-21T17:41:39.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.051+0000 7fbd296de640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd240728f0 0x7fbd240b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:39.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.051+0000 7fbd29edf640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd24145c70 0x7fbd240b8040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c036040 tx=0x7fbd1c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.051+0000 7f4c537fe640 1 -- 172.21.15.3:0/3597917067 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f4c5c01a070 con 0x7f4c6c075540 2024-05-21T17:41:39.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.051+0000 7fbd19ffb640 1 -- 172.21.15.3:0/1623917862 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c043020 con 0x7fbd24145c70 2024-05-21T17:41:39.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.051+0000 7f4c537fe640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:39.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.052+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd240b8810 con 0x7fbd24145c70 2024-05-21T17:41:39.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.052+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf8069a90 con 0x7fccf806d350 2024-05-21T17:41:39.088 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.052+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd241d2580 con 0x7fbd24145c70 2024-05-21T17:41:39.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7f4c537fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:39.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7f4c537fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:39.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7f4c537fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:39.089 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7f4c537fe640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c580af3b0 0x7f4c580b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7fb4567fc640 1 -- 172.21.15.3:0/837002327 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7fb45803f040 con 0x7fb4600b8e50 2024-05-21T17:41:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7fb4567fc640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.052+0000 7fbd19ffb640 1 -- 172.21.15.3:0/1623917862 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c03e450 con 0x7fbd24145c70 2024-05-21T17:41:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.052+0000 7fbd19ffb640 1 -- 172.21.15.3:0/1623917862 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c03e860 con 0x7fbd24145c70 2024-05-21T17:41:39.090 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7f4c6a7fc640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c580af3b0 0x7f4c580b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7f4c537fe640 1 -- 172.21.15.3:0/3597917067 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f4c5c0dcc30 con 0x7f4c6c075540 2024-05-21T17:41:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd240b9990 con 0x7fbd24145c70 2024-05-21T17:41:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7fccee7fc640 1 -- 172.21.15.3:0/3826963854 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7fccf401e460 con 0x7fccf806d350 2024-05-21T17:41:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.053+0000 7fccee7fc640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:39.091 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fccee7fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fccee7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fccee7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fccee7fc640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fccdc0af3b0 0x7fccdc0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7f4c6a7fc640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c580af3b0 0x7f4c580b1870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0aa850 tx=0x7f4c60009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fb4567fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:39.092 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fb4567fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fb4567fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fb4567fc640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb4440af210 0x7fb4440b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fccee7fc640 1 -- 172.21.15.3:0/3826963854 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7fccf40dd9e0 con 0x7fccf806d350 2024-05-21T17:41:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fb465d7a640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb4440af210 0x7fb4440b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.093 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fb4567fc640 1 -- 172.21.15.3:0/837002327 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7fb458103cc0 con 0x7fb4600b8e50 2024-05-21T17:41:39.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.054+0000 7fccfd1bc640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fccdc0af3b0 0x7fccdc0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.055+0000 7fccfd1bc640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fccdc0af3b0 0x7fccdc0b1870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fccf80bab30 tx=0x7fcce8009240 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.094 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.055+0000 7fb465d7a640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb4440af210 0x7fb4440b16d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009630 tx=0x7fb45c009240 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.055+0000 7fbd19ffb640 1 -- 172.21.15.3:0/1623917862 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7fbd1c005d10 con 0x7fbd24145c70 2024-05-21T17:41:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.055+0000 7fbd19ffb640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.056+0000 7fbd19ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.056+0000 7fbd19ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.056+0000 7fbd19ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.056+0000 7fbd19ffb640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbd080af260 0x7fbd080b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.056+0000 7fbd296de640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbd080af260 0x7fbd080b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.056+0000 7fbd19ffb640 1 -- 172.21.15.3:0/1623917862 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7fbd1c0ffbb0 con 0x7fbd24145c70 2024-05-21T17:41:39.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.057+0000 7fbd296de640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbd080af260 0x7fbd080b1720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd20007fd0 tx=0x7fbd20009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:39.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.058+0000 7f4c537fe640 1 -- 172.21.15.3:0/3597917067 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4c5c0a6940 con 0x7f4c6c075540 2024-05-21T17:41:39.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.059+0000 7fbd19ffb640 1 -- 172.21.15.3:0/1623917862 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbd1c0c98c0 con 0x7fbd24145c70 2024-05-21T17:41:39.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.062+0000 7fccee7fc640 1 -- 172.21.15.3:0/3826963854 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fccf40a7770 con 0x7fccf806d350 2024-05-21T17:41:39.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.070+0000 7fb4567fc640 1 -- 172.21.15.3:0/837002327 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb4601423f0 con 0x7fb4600b8e50 2024-05-21T17:41:39.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.254+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbd24069290 con 0x7fbd24145c70 2024-05-21T17:41:39.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.255+0000 7fbd19ffb640 1 -- 172.21.15.3:0/1623917862 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd1c0c91c0 con 0x7fbd24145c70 2024-05-21T17:41:39.256 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2024-05-21T17:41:39.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 10 mgrc shutdown 2024-05-21T17:41:39.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbd080af260 msgr2=0x7fbd080b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbd080af260 0x7fbd080b1720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd20007fd0 tx=0x7fbd20009040 comp rx=0 tx=0).stop 2024-05-21T17:41:39.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd24145c70 msgr2=0x7fbd240b8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd24145c70 0x7fbd240b8040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c036040 tx=0x7fbd1c004a40 comp rx=0 tx=0).stop 2024-05-21T17:41:39.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 shutdown_connections 2024-05-21T17:41:39.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbd080af260 0x7fbd080b1720 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd24145c70 0x7fbd240b8040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd240b60b0 0x7fbd240b7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 --2- 172.21.15.3:0/1623917862 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbd240728f0 0x7fbd240b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 >> 172.21.15.3:0/1623917862 conn(0x7fbd240713c0 msgr2=0x7fbd240681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:39.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 shutdown_connections 2024-05-21T17:41:39.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.257+0000 7fbd2b942640 1 -- 172.21.15.3:0/1623917862 wait complete. 2024-05-21T17:41:39.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.261+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fccf8075db0 con 0x7fccf806d350 2024-05-21T17:41:39.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.262+0000 7fccee7fc640 1 -- 172.21.15.3:0/3826963854 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fccf40a7070 con 0x7fccf806d350 2024-05-21T17:41:39.263 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2024-05-21T17:41:39.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 10 mgrc shutdown 2024-05-21T17:41:39.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fccdc0af3b0 msgr2=0x7fccdc0b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fccdc0af3b0 0x7fccdc0b1870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fccf80bab30 tx=0x7fcce8009240 comp rx=0 tx=0).stop 2024-05-21T17:41:39.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 msgr2=0x7fccf81d59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 0x7fccf81d59d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fccf400d830 tx=0x7fccf400dcf0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 shutdown_connections 2024-05-21T17:41:39.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fccdc0af3b0 0x7fccdc0b1870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fccf814a5c0 0x7fccf80a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fccf80a9880 0x7fccf806f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 --2- 172.21.15.3:0/3826963854 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fccf806d350 0x7fccf81d59d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 >> 172.21.15.3:0/3826963854 conn(0x7fccf80bbc20 msgr2=0x7fccf80bac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:39.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 shutdown_connections 2024-05-21T17:41:39.266 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.264+0000 7fccffc21640 1 -- 172.21.15.3:0/3826963854 wait complete. 2024-05-21T17:41:39.271 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.3 2024-05-21T17:41:39.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.1 2024-05-21T17:41:39.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.307+0000 7f4c71705640 1 -- 172.21.15.3:0/3597917067 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4c6c0a5800 con 0x7f4c6c075540 2024-05-21T17:41:39.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.307+0000 7f4c537fe640 1 -- 172.21.15.3:0/3597917067 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4c5c0a6240 con 0x7f4c6c075540 2024-05-21T17:41:39.308 INFO:teuthology.orchestra.run.smithi003.stdout:38654705668 2024-05-21T17:41:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.309+0000 7f4c71705640 10 mgrc shutdown 2024-05-21T17:41:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.309+0000 7f4c71705640 1 -- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c580af3b0 msgr2=0x7f4c580b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.309+0000 7f4c71705640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c580af3b0 0x7f4c580b1870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0aa850 tx=0x7f4c60009040 comp rx=0 tx=0).stop 2024-05-21T17:41:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.309+0000 7f4c71705640 1 -- 172.21.15.3:0/3597917067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c075540 msgr2=0x7f4c6c073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.309+0000 7f4c71705640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c075540 0x7f4c6c073f70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00e9b0 tx=0x7f4c5c00ee70 comp rx=0 tx=0).stop 2024-05-21T17:41:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.309+0000 7f4c71705640 1 -- 172.21.15.3:0/3597917067 shutdown_connections 2024-05-21T17:41:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.309+0000 7f4c71705640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c580af3b0 0x7f4c580b1870 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.309+0000 7f4c71705640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c6c145800 0x7f4c6c0b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.310+0000 7f4c71705640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c6c0a94c0 0x7f4c6c0b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.310+0000 7f4c71705640 1 --2- 172.21.15.3:0/3597917067 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c6c075540 0x7f4c6c073f70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.310+0000 7f4c71705640 1 -- 172.21.15.3:0/3597917067 >> 172.21.15.3:0/3597917067 conn(0x7f4c6c0a74a0 msgr2=0x7f4c6c0681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:39.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.310+0000 7f4c71705640 1 -- 172.21.15.3:0/3597917067 shutdown_connections 2024-05-21T17:41:39.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.310+0000 7f4c71705640 1 -- 172.21.15.3:0/3597917067 wait complete. 2024-05-21T17:41:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.316+0000 7fb467fde640 1 -- 172.21.15.3:0/837002327 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb4601cd9f0 con 0x7fb4600b8e50 2024-05-21T17:41:39.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.317+0000 7fb4567fc640 1 -- 172.21.15.3:0/837002327 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb4601cd9f0 con 0x7fb4600b8e50 2024-05-21T17:41:39.318 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2024-05-21T17:41:39.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 10 mgrc shutdown 2024-05-21T17:41:39.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 -- 172.21.15.3:0/837002327 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb4440af210 msgr2=0x7fb4440b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb4440af210 0x7fb4440b16d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009630 tx=0x7fb45c009240 comp rx=0 tx=0).stop 2024-05-21T17:41:39.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 -- 172.21.15.3:0/837002327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600b8e50 msgr2=0x7fb4601457c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:39.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600b8e50 0x7fb4601457c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb458030ca0 tx=0x7fb458004a40 comp rx=0 tx=0).stop 2024-05-21T17:41:39.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 -- 172.21.15.3:0/837002327 shutdown_connections 2024-05-21T17:41:39.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb4440af210 0x7fb4440b16d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb4600b8e50 0x7fb4601457c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb4600a5a70 0x7fb4600ac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 --2- 172.21.15.3:0/837002327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb46006c410 0x7fb46013e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:39.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 -- 172.21.15.3:0/837002327 >> 172.21.15.3:0/837002327 conn(0x7fb46006d700 msgr2=0x7fb4600b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:39.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 -- 172.21.15.3:0/837002327 shutdown_connections 2024-05-21T17:41:39.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:39.319+0000 7fb467fde640 1 -- 172.21.15.3:0/837002327 wait complete. 2024-05-21T17:41:39.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2024-05-21T17:41:39.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.2 2024-05-21T17:41:40.271 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-21T17:41:40.277 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-21T17:41:40.323 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-21T17:41:40.331 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-21T17:41:40.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.390+0000 7fbff44f5640 1 Processor -- start 2024-05-21T17:41:40.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.390+0000 7fbff44f5640 1 -- start start 2024-05-21T17:41:40.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff44f5640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec132820 0x7fbfec0735e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff44f5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbfec073b20 0x7fbfec0ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff44f5640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbfec06dce0 0x7fbfec0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff44f5640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fbfec074540 con 0x7fbfec06dce0 2024-05-21T17:41:40.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff44f5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fbfec146550 con 0x7fbfec073b20 2024-05-21T17:41:40.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff44f5640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fbfec146810 con 0x7fbfec132820 2024-05-21T17:41:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff2291640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec132820 0x7fbfec0735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff2291640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec132820 0x7fbfec0735e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41020/0 (socket says 172.21.15.3:41020) 2024-05-21T17:41:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff2a92640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fbfec06dce0 0x7fbfec0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff2291640 1 -- 172.21.15.3:0/3608953630 learned_addr learned my addr 172.21.15.3:0/3608953630 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff1a90640 1 --2- 172.21.15.3:0/3608953630 >> v2:172.21.15.203:3300/0 conn(0x7fbfec073b20 0x7fbfec0ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff2291640 1 -- 172.21.15.3:0/3608953630 >> v2:172.21.15.203:3300/0 conn(0x7fbfec073b20 msgr2=0x7fbfec0ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff2291640 1 --2- 172.21.15.3:0/3608953630 >> v2:172.21.15.203:3300/0 conn(0x7fbfec073b20 0x7fbfec0ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff2291640 1 -- 172.21.15.3:0/3608953630 >> v2:172.21.15.3:3300/0 conn(0x7fbfec06dce0 msgr2=0x7fbfec0a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff2291640 1 --2- 172.21.15.3:0/3608953630 >> v2:172.21.15.3:3300/0 conn(0x7fbfec06dce0 0x7fbfec0a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.392+0000 7fbff2291640 1 -- 172.21.15.3:0/3608953630 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfec0749d0 con 0x7fbfec132820 2024-05-21T17:41:40.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.394+0000 7fbff2291640 1 --2- 172.21.15.3:0/3608953630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec132820 0x7fbfec0735e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0aa540 tx=0x7fbfe802e050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c1134958ae36acf server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.394+0000 7fbff128f640 1 -- 172.21.15.3:0/3608953630 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe803b070 con 0x7fbfec132820 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.394+0000 7fbff128f640 1 -- 172.21.15.3:0/3608953630 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe802ee00 con 0x7fbfec132820 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.394+0000 7fbff128f640 1 -- 172.21.15.3:0/3608953630 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8036660 con 0x7fbfec132820 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.395+0000 7fbff44f5640 1 -- 172.21.15.3:0/3608953630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec132820 msgr2=0x7fbfec0735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.395+0000 7fbff44f5640 1 --2- 172.21.15.3:0/3608953630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec132820 0x7fbfec0735e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0aa540 tx=0x7fbfe802e050 comp rx=0 tx=0).stop 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.395+0000 7fbff44f5640 1 -- 172.21.15.3:0/3608953630 shutdown_connections 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.395+0000 7fbff44f5640 1 --2- 172.21.15.3:0/3608953630 >> v2:172.21.15.3:3300/0 conn(0x7fbfec06dce0 0x7fbfec0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.395+0000 7fbff44f5640 1 --2- 172.21.15.3:0/3608953630 >> v2:172.21.15.203:3300/0 conn(0x7fbfec073b20 0x7fbfec0ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.395+0000 7fbff44f5640 1 --2- 172.21.15.3:0/3608953630 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec132820 0x7fbfec0735e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.395+0000 7fbff44f5640 1 -- 172.21.15.3:0/3608953630 >> 172.21.15.3:0/3608953630 conn(0x7fbfec0713e0 msgr2=0x7fbfec06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:40.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.395+0000 7fbff44f5640 1 -- 172.21.15.3:0/3608953630 shutdown_connections 2024-05-21T17:41:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.395+0000 7fbff44f5640 1 -- 172.21.15.3:0/3608953630 wait complete. 2024-05-21T17:41:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.397+0000 7fbff44f5640 1 Processor -- start 2024-05-21T17:41:40.398 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.397+0000 7fbff44f5640 1 -- start start 2024-05-21T17:41:40.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff44f5640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec06dce0 0x7fbfec0ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff44f5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfec073b20 0x7fbfec1ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff44f5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfec132820 0x7fbfec1cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff44f5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbfec132c00 con 0x7fbfec073b20 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff44f5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbfec139610 con 0x7fbfec132820 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff44f5640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fbfec132d80 con 0x7fbfec06dce0 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff2291640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec06dce0 0x7fbfec0ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff2291640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec06dce0 0x7fbfec0ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41024/0 (socket says 172.21.15.3:41024) 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff2291640 1 -- 172.21.15.3:0/1166685580 learned_addr learned my addr 172.21.15.3:0/1166685580 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff1a90640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfec073b20 0x7fbfec1ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.398+0000 7fbff2a92640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfec132820 0x7fbfec1cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.399+0000 7fbff2291640 1 -- 172.21.15.3:0/1166685580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfec132820 msgr2=0x7fbfec1cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.399+0000 7fbff2291640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfec132820 0x7fbfec1cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.399+0000 7fbff2291640 1 -- 172.21.15.3:0/1166685580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfec073b20 msgr2=0x7fbfec1ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.399+0000 7fbff2291640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfec073b20 0x7fbfec1ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.399+0000 7fbff2291640 1 -- 172.21.15.3:0/1166685580 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfec1cc620 con 0x7fbfec06dce0 2024-05-21T17:41:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.399+0000 7fbff2291640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec06dce0 0x7fbfec0ad7b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfe802e090 tx=0x7fbfe80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.399+0000 7fbfe2ffd640 1 -- 172.21.15.3:0/1166685580 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8045030 con 0x7fbfec06dce0 2024-05-21T17:41:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.399+0000 7fbfe2ffd640 1 -- 172.21.15.3:0/1166685580 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfe802ec00 con 0x7fbfec06dce0 2024-05-21T17:41:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.399+0000 7fbfe2ffd640 1 -- 172.21.15.3:0/1166685580 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe803b040 con 0x7fbfec06dce0 2024-05-21T17:41:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.399+0000 7fbff44f5640 1 -- 172.21.15.3:0/1166685580 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfec130ca0 con 0x7fbfec06dce0 2024-05-21T17:41:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.400+0000 7fbff44f5640 1 -- 172.21.15.3:0/1166685580 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfec133fe0 con 0x7fbfec06dce0 2024-05-21T17:41:40.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.401+0000 7fbff44f5640 1 -- 172.21.15.3:0/1166685580 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfec069290 con 0x7fbfec06dce0 2024-05-21T17:41:40.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.403+0000 7fbfe2ffd640 1 -- 172.21.15.3:0/1166685580 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7fbfe8048070 con 0x7fbfec06dce0 2024-05-21T17:41:40.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.403+0000 7fbfe2ffd640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:40.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.404+0000 7fbfe2ffd640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:40.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.404+0000 7fbfe2ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:40.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.404+0000 7fbfe2ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:40.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.404+0000 7fbfe2ffd640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbfd00af260 0x7fbfd00b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.405+0000 7fbfe2ffd640 1 -- 172.21.15.3:0/1166685580 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7fbfe8100e10 con 0x7fbfec06dce0 2024-05-21T17:41:40.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.405+0000 7fbff1a90640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbfd00af260 0x7fbfd00b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.405+0000 7fbff1a90640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbfd00af260 0x7fbfd00b1720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbfec135e20 tx=0x7fbfdc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.409+0000 7fbfe2ffd640 1 -- 172.21.15.3:0/1166685580 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbfe804f070 con 0x7fbfec06dce0 2024-05-21T17:41:40.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.432+0000 7f1844283640 1 Processor -- start 2024-05-21T17:41:40.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.432+0000 7f1844283640 1 -- start start 2024-05-21T17:41:40.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.433+0000 7f1844283640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0aa2f0 0x7f183c146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.433+0000 7f1844283640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183c06c7a0 con 0x7f183c147130 2024-05-21T17:41:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.433+0000 7f1844283640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183c0adfb0 con 0x7f183c0ac530 2024-05-21T17:41:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.433+0000 7f1844283640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f183c0a54b0 con 0x7f183c0aa2f0 2024-05-21T17:41:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.433+0000 7f184181e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0aa2f0 0x7f183c146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.434+0000 7f1842820640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f183c147130 0x7f183c150cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60710/0 (socket says 172.21.15.3:60710) 2024-05-21T17:41:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.434+0000 7f1842820640 1 -- 172.21.15.3:0/3479934023 learned_addr learned my addr 172.21.15.3:0/3479934023 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:40.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.434+0000 7f184101d640 1 -- 172.21.15.3:0/3479934023 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171386149 0 0) 0x7f183c06c7a0 con 0x7f183c147130 2024-05-21T17:41:40.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.434+0000 7f184181e640 1 -- 172.21.15.3:0/3479934023 >> v1:172.21.15.203:6789/0 conn(0x7f183c0ac530 legacy=0x7f183c0ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.434+0000 7f184181e640 1 -- 172.21.15.3:0/3479934023 >> v1:172.21.15.3:6789/0 conn(0x7f183c147130 legacy=0x7f183c150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.434+0000 7f184181e640 1 -- 172.21.15.3:0/3479934023 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c0ac910 con 0x7f183c0aa2f0 2024-05-21T17:41:40.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.435+0000 7f184181e640 1 --2- 172.21.15.3:0/3479934023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0aa2f0 0x7f183c146bf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f182c009750 tx=0x7f182c02b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc5a2d5e174c4dce server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.435+0000 7f184101d640 1 -- 172.21.15.3:0/3479934023 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c037070 con 0x7f183c0aa2f0 2024-05-21T17:41:40.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.435+0000 7f184101d640 1 -- 172.21.15.3:0/3479934023 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f182c030710 con 0x7f183c0aa2f0 2024-05-21T17:41:40.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.435+0000 7f184101d640 1 -- 172.21.15.3:0/3479934023 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c030ae0 con 0x7f183c0aa2f0 2024-05-21T17:41:40.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.436+0000 7f1844283640 1 -- 172.21.15.3:0/3479934023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0aa2f0 msgr2=0x7f183c146bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.436+0000 7f1844283640 1 --2- 172.21.15.3:0/3479934023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0aa2f0 0x7f183c146bf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f182c009750 tx=0x7f182c02b600 comp rx=0 tx=0).stop 2024-05-21T17:41:40.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.436+0000 7f1844283640 1 -- 172.21.15.3:0/3479934023 shutdown_connections 2024-05-21T17:41:40.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.436+0000 7f1844283640 1 --2- 172.21.15.3:0/3479934023 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0aa2f0 0x7f183c146bf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.436+0000 7f1844283640 1 -- 172.21.15.3:0/3479934023 >> 172.21.15.3:0/3479934023 conn(0x7f183c0bba40 msgr2=0x7f183c0bbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:40.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.436+0000 7f1844283640 1 -- 172.21.15.3:0/3479934023 shutdown_connections 2024-05-21T17:41:40.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.436+0000 7f1844283640 1 -- 172.21.15.3:0/3479934023 wait complete. 2024-05-21T17:41:40.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.437+0000 7f1844283640 1 Processor -- start 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.437+0000 7f1844283640 1 -- start start 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f1844283640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f183c0aa2f0 0x7f183c09dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f1844283640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0ac530 0x7f183c09c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f1844283640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c147130 0x7f183c09cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f1844283640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f183c06cde0 con 0x7f183c147130 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f1844283640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f183c06cc60 con 0x7f183c0aa2f0 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f1844283640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f183c0a82b0 con 0x7f183c0ac530 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f1842820640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c147130 0x7f183c09cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f1842820640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c147130 0x7f183c09cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51570/0 (socket says 172.21.15.3:51570) 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f1842820640 1 -- 172.21.15.3:0/858042564 learned_addr learned my addr 172.21.15.3:0/858042564 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f184181e640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0ac530 0x7f183c09c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.438+0000 7f184201f640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f183c0aa2f0 0x7f183c09dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1842820640 1 -- 172.21.15.3:0/858042564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0ac530 msgr2=0x7f183c09c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1842820640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0ac530 0x7f183c09c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1842820640 1 -- 172.21.15.3:0/858042564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f183c0aa2f0 msgr2=0x7f183c09dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1842820640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f183c0aa2f0 0x7f183c09dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1842820640 1 -- 172.21.15.3:0/858042564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c1d0b30 con 0x7f183c147130 2024-05-21T17:41:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f184181e640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0ac530 0x7f183c09c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:40.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1842820640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c147130 0x7f183c09cad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f183400c6b0 tx=0x7f183400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1832ffd640 1 -- 172.21.15.3:0/858042564 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834018070 con 0x7f183c147130 2024-05-21T17:41:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1832ffd640 1 -- 172.21.15.3:0/858042564 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1834004da0 con 0x7f183c147130 2024-05-21T17:41:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1832ffd640 1 -- 172.21.15.3:0/858042564 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183400a880 con 0x7f183c147130 2024-05-21T17:41:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1844283640 1 -- 172.21.15.3:0/858042564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c1d0df0 con 0x7f183c147130 2024-05-21T17:41:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.439+0000 7f1844283640 1 -- 172.21.15.3:0/858042564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c1d2100 con 0x7f183c147130 2024-05-21T17:41:40.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.440+0000 7f1844283640 1 -- 172.21.15.3:0/858042564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c069290 con 0x7f183c147130 2024-05-21T17:41:40.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.442+0000 7f1832ffd640 1 -- 172.21.15.3:0/858042564 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f183400aa20 con 0x7f183c147130 2024-05-21T17:41:40.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.442+0000 7f1832ffd640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.443+0000 7f1832ffd640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.443+0000 7f1832ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.443+0000 7f1832ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.443+0000 7f1832ffd640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f18200af1e0 0x7f18200b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.443+0000 7f1832ffd640 1 -- 172.21.15.3:0/858042564 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f18340dbe30 con 0x7f183c147130 2024-05-21T17:41:40.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.443+0000 7f184201f640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f18200af1e0 0x7f18200b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.444+0000 7f184201f640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f18200af1e0 0x7f18200b16a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1838000f60 tx=0x7f1838009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.447+0000 7f1832ffd640 1 -- 172.21.15.3:0/858042564 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f183404f990 con 0x7f183c147130 2024-05-21T17:41:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.473+0000 7fb6b11b8640 1 Processor -- start 2024-05-21T17:41:40.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.473+0000 7fb6b11b8640 1 -- start start 2024-05-21T17:41:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.474+0000 7fb6b11b8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6ac0a6f50 0x7fb6ac0a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.474+0000 7fb6b11b8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac146d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.474+0000 7fb6b11b8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6ac147250 con 0x7fb6ac149850 2024-05-21T17:41:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.474+0000 7fb6b11b8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb6ac074980 con 0x7fb6ac0a6f50 2024-05-21T17:41:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.474+0000 7fb6b11b8640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ac1465d0 con 0x7fb6ac146890 2024-05-21T17:41:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.474+0000 7fb6aa59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.474+0000 7fb6aa59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac146d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41044/0 (socket says 172.21.15.3:41044) 2024-05-21T17:41:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.474+0000 7fb6aa59c640 1 -- 172.21.15.3:0/670263907 learned_addr learned my addr 172.21.15.3:0/670263907 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.474+0000 7fb6aad9d640 1 --2- 172.21.15.3:0/670263907 >> v2:172.21.15.203:3300/0 conn(0x7fb6ac0a6f50 0x7fb6ac0a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.475+0000 7fb6aa59c640 1 -- 172.21.15.3:0/670263907 >> v2:172.21.15.203:3300/0 conn(0x7fb6ac0a6f50 msgr2=0x7fb6ac0a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:40.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.475+0000 7fb6aa59c640 1 --2- 172.21.15.3:0/670263907 >> v2:172.21.15.203:3300/0 conn(0x7fb6ac0a6f50 0x7fb6ac0a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.475+0000 7fb6aa59c640 1 -- 172.21.15.3:0/670263907 >> v1:172.21.15.203:6789/0 conn(0x7fb6ac149850 legacy=0x7fb6ac149ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:40.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.475+0000 7fb6aa59c640 1 -- 172.21.15.3:0/670263907 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac146290 con 0x7fb6ac146890 2024-05-21T17:41:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.475+0000 7fb6aa59c640 1 --2- 172.21.15.3:0/670263907 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac146d10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb694009870 tx=0x7fb69402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=896dfbdf81b0859e server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.476+0000 7fb6a9d9b640 1 -- 172.21.15.3:0/670263907 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6940317c0 con 0x7fb6ac146890 2024-05-21T17:41:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.476+0000 7fb6a9d9b640 1 -- 172.21.15.3:0/670263907 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb694037070 con 0x7fb6ac146890 2024-05-21T17:41:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.476+0000 7fb6a9d9b640 1 -- 172.21.15.3:0/670263907 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694031b80 con 0x7fb6ac146890 2024-05-21T17:41:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.476+0000 7fb6b11b8640 1 -- 172.21.15.3:0/670263907 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 msgr2=0x7fb6ac146d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.476+0000 7fb6b11b8640 1 --2- 172.21.15.3:0/670263907 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac146d10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb694009870 tx=0x7fb69402b860 comp rx=0 tx=0).stop 2024-05-21T17:41:40.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.476+0000 7fb6b11b8640 1 -- 172.21.15.3:0/670263907 shutdown_connections 2024-05-21T17:41:40.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.476+0000 7fb6b11b8640 1 --2- 172.21.15.3:0/670263907 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac146d10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.476+0000 7fb6b11b8640 1 --2- 172.21.15.3:0/670263907 >> v2:172.21.15.203:3300/0 conn(0x7fb6ac0a6f50 0x7fb6ac0a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.476+0000 7fb6b11b8640 1 -- 172.21.15.3:0/670263907 >> 172.21.15.3:0/670263907 conn(0x7fb6ac0a8c40 msgr2=0x7fb6ac0a9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:40.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.477+0000 7fb6b11b8640 1 -- 172.21.15.3:0/670263907 shutdown_connections 2024-05-21T17:41:40.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.477+0000 7fb6b11b8640 1 -- 172.21.15.3:0/670263907 wait complete. 2024-05-21T17:41:40.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.478+0000 7fb6b11b8640 1 Processor -- start 2024-05-21T17:41:40.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.478+0000 7fb6b11b8640 1 -- start start 2024-05-21T17:41:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.479+0000 7fb6b11b8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac0a6f50 0x7fb6ac0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.479+0000 7fb6b11b8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac09ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.479+0000 7fb6aad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac0a6f50 0x7fb6ac0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.479+0000 7fb6aad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac0a6f50 0x7fb6ac0b7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51576/0 (socket says 172.21.15.3:51576) 2024-05-21T17:41:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.479+0000 7fb6aad9d640 1 -- 172.21.15.3:0/2519315701 learned_addr learned my addr 172.21.15.3:0/2519315701 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.479+0000 7fb6b11b8640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6ac149850 0x7fb6ac1403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.479+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac0a74b0 con 0x7fb6ac0a6f50 2024-05-21T17:41:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.479+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac0a7330 con 0x7fb6ac149850 2024-05-21T17:41:40.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.479+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ac0a6860 con 0x7fb6ac146890 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.479+0000 7fb6aa59c640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac09ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6ab59e640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6ac149850 0x7fb6ac1403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6aa59c640 1 -- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6ac149850 msgr2=0x7fb6ac1403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6aa59c640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6ac149850 0x7fb6ac1403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6aa59c640 1 -- 172.21.15.3:0/2519315701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac0a6f50 msgr2=0x7fb6ac0b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6aa59c640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac0a6f50 0x7fb6ac0b7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6aa59c640 1 -- 172.21.15.3:0/2519315701 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac141030 con 0x7fb6ac146890 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6aa59c640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac09ef80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb694033000 tx=0x7fb6940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6a37fe640 1 -- 172.21.15.3:0/2519315701 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694032650 con 0x7fb6ac146890 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6a37fe640 1 -- 172.21.15.3:0/2519315701 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb694032830 con 0x7fb6ac146890 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ac1412f0 con 0x7fb6ac146890 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.480+0000 7fb6a37fe640 1 -- 172.21.15.3:0/2519315701 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694032c20 con 0x7fb6ac146890 2024-05-21T17:41:40.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.481+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ac09f4c0 con 0x7fb6ac146890 2024-05-21T17:41:40.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.481+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ac069290 con 0x7fb6ac146890 2024-05-21T17:41:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.484+0000 7fb6a37fe640 1 -- 172.21.15.3:0/2519315701 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7fb69404c060 con 0x7fb6ac146890 2024-05-21T17:41:40.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.484+0000 7fb6a37fe640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.485+0000 7fb6a37fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.485+0000 7fb6a37fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.485+0000 7fb6a37fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.485+0000 7fb6a37fe640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb68c0af210 0x7fb68c0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.485+0000 7fb6a37fe640 1 -- 172.21.15.3:0/2519315701 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7fb6940fe6f0 con 0x7fb6ac146890 2024-05-21T17:41:40.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.485+0000 7fb6aad9d640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb68c0af210 0x7fb68c0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.486+0000 7fb6aad9d640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb68c0af210 0x7fb68c0b16d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb69c003bc0 tx=0x7fb69c009970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.488+0000 7fb6a37fe640 1 -- 172.21.15.3:0/2519315701 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb694080990 con 0x7fb6ac146890 2024-05-21T17:41:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.507+0000 7f2db56f5640 1 Processor -- start 2024-05-21T17:41:40.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.507+0000 7f2db56f5640 1 -- start start 2024-05-21T17:41:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.508+0000 7f2db56f5640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db00ac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.508+0000 7f2db56f5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2db0139470 0x7f2db01353d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.508+0000 7f2db56f5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2db0072080 con 0x7f2db0132b90 2024-05-21T17:41:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.508+0000 7f2db56f5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f2db00acd50 con 0x7f2db0139470 2024-05-21T17:41:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.508+0000 7f2db56f5640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f2db0073e00 con 0x7f2db01327b0 2024-05-21T17:41:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.508+0000 7f2daeffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db00ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.508+0000 7f2daeffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db00ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41070/0 (socket says 172.21.15.3:41070) 2024-05-21T17:41:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.508+0000 7f2daf7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f2db0132b90 0x7f2db00736f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60716/0 (socket says 172.21.15.3:60716) 2024-05-21T17:41:40.510 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.508+0000 7f2daeffd640 1 -- 172.21.15.3:0/480796377 learned_addr learned my addr 172.21.15.3:0/480796377 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.508+0000 7f2dae7fc640 1 --2- 172.21.15.3:0/480796377 >> v2:172.21.15.203:3300/0 conn(0x7f2db0139470 0x7f2db01353d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.509+0000 7f2daeffd640 1 -- 172.21.15.3:0/480796377 >> v2:172.21.15.203:3300/0 conn(0x7f2db0139470 msgr2=0x7f2db01353d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.509+0000 7f2daeffd640 1 --2- 172.21.15.3:0/480796377 >> v2:172.21.15.203:3300/0 conn(0x7f2db0139470 0x7f2db01353d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.509+0000 7f2daeffd640 1 -- 172.21.15.3:0/480796377 >> v1:172.21.15.3:6789/0 conn(0x7f2db0132b90 legacy=0x7f2db00736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.509+0000 7f2daeffd640 1 -- 172.21.15.3:0/480796377 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db0135c90 con 0x7f2db01327b0 2024-05-21T17:41:40.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.509+0000 7f2dae7fc640 1 --2- 172.21.15.3:0/480796377 >> v2:172.21.15.203:3300/0 conn(0x7f2db0139470 0x7f2db01353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.510+0000 7f2daeffd640 1 --2- 172.21.15.3:0/480796377 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db00ac460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009870 tx=0x7f2da002b740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a53f54022c4e0545 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.510+0000 7f2dadffb640 1 -- 172.21.15.3:0/480796377 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0037070 con 0x7f2db01327b0 2024-05-21T17:41:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.510+0000 7f2dadffb640 1 -- 172.21.15.3:0/480796377 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2da0030710 con 0x7f2db01327b0 2024-05-21T17:41:40.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.510+0000 7f2dadffb640 1 -- 172.21.15.3:0/480796377 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0030ae0 con 0x7f2db01327b0 2024-05-21T17:41:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.511+0000 7f2db56f5640 1 -- 172.21.15.3:0/480796377 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 msgr2=0x7f2db00ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.511+0000 7f2db56f5640 1 --2- 172.21.15.3:0/480796377 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db00ac460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009870 tx=0x7f2da002b740 comp rx=0 tx=0).stop 2024-05-21T17:41:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.511+0000 7f2db56f5640 1 -- 172.21.15.3:0/480796377 shutdown_connections 2024-05-21T17:41:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.511+0000 7f2db56f5640 1 --2- 172.21.15.3:0/480796377 >> v2:172.21.15.203:3300/0 conn(0x7f2db0139470 0x7f2db01353d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.511+0000 7f2db56f5640 1 --2- 172.21.15.3:0/480796377 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db00ac460 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.511+0000 7f2db56f5640 1 -- 172.21.15.3:0/480796377 >> 172.21.15.3:0/480796377 conn(0x7f2db00713c0 msgr2=0x7f2db006d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.511+0000 7f2db56f5640 1 -- 172.21.15.3:0/480796377 shutdown_connections 2024-05-21T17:41:40.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.511+0000 7f2db56f5640 1 -- 172.21.15.3:0/480796377 wait complete. 2024-05-21T17:41:40.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.513+0000 7f2db56f5640 1 Processor -- start 2024-05-21T17:41:40.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.513+0000 7f2db56f5640 1 -- start start 2024-05-21T17:41:40.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.514+0000 7f2db56f5640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db0134d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.514+0000 7f2db56f5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db0132b90 0x7f2db00b7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.514+0000 7f2db56f5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2db0139470 0x7f2db01427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.514+0000 7f2db56f5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2db006c410 con 0x7f2db0139470 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.514+0000 7f2db56f5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2db0135910 con 0x7f2db0132b90 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.514+0000 7f2db56f5640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f2db006c590 con 0x7f2db01327b0 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.514+0000 7f2daeffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db0134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.514+0000 7f2daeffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db0134d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41076/0 (socket says 172.21.15.3:41076) 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.514+0000 7f2daeffd640 1 -- 172.21.15.3:0/1187218647 learned_addr learned my addr 172.21.15.3:0/1187218647 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2dae7fc640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db0132b90 0x7f2db00b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2daf7fe640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2db0139470 0x7f2db01427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2daeffd640 1 -- 172.21.15.3:0/1187218647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db0132b90 msgr2=0x7f2db00b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2daeffd640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db0132b90 0x7f2db00b7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2daeffd640 1 -- 172.21.15.3:0/1187218647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2db0139470 msgr2=0x7f2db01427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2daeffd640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2db0139470 0x7f2db01427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2daeffd640 1 -- 172.21.15.3:0/1187218647 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db00b7bc0 con 0x7f2db01327b0 2024-05-21T17:41:40.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2daeffd640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db0134d50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2da002b780 tx=0x7f2da00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2d977fe640 1 -- 172.21.15.3:0/1187218647 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0041020 con 0x7f2db01327b0 2024-05-21T17:41:40.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2d977fe640 1 -- 172.21.15.3:0/1187218647 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2da003c870 con 0x7f2db01327b0 2024-05-21T17:41:40.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2db56f5640 1 -- 172.21.15.3:0/1187218647 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db00b5cb0 con 0x7f2db01327b0 2024-05-21T17:41:40.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2d977fe640 1 -- 172.21.15.3:0/1187218647 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0037040 con 0x7f2db01327b0 2024-05-21T17:41:40.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.515+0000 7f2db56f5640 1 -- 172.21.15.3:0/1187218647 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db0142f00 con 0x7f2db01327b0 2024-05-21T17:41:40.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.517+0000 7f2db56f5640 1 -- 172.21.15.3:0/1187218647 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db01431f0 con 0x7f2db01327b0 2024-05-21T17:41:40.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.520+0000 7f2d977fe640 1 -- 172.21.15.3:0/1187218647 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f2da0044020 con 0x7f2db01327b0 2024-05-21T17:41:40.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.520+0000 7f2d977fe640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.521+0000 7f2d977fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.521+0000 7f2d977fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.521+0000 7f2d977fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.521+0000 7f2d977fe640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f2d9c0af260 0x7f2d9c0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.521+0000 7f2d977fe640 1 -- 172.21.15.3:0/1187218647 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f2da00fe6e0 con 0x7f2db01327b0 2024-05-21T17:41:40.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.521+0000 7f2dae7fc640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f2d9c0af260 0x7f2d9c0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.522+0000 7f2dae7fc640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f2d9c0af260 0x7f2d9c0b1720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009560 tx=0x7f2da4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.523+0000 7f2d977fe640 1 -- 172.21.15.3:0/1187218647 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2db01431f0 con 0x7f2db01327b0 2024-05-21T17:41:40.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.646+0000 7f1844283640 1 -- 172.21.15.3:0/858042564 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f183c0a7980 con 0x7f183c147130 2024-05-21T17:41:40.648 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.647+0000 7f1832ffd640 1 -- 172.21.15.3:0/858042564 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f18340a5570 con 0x7f183c147130 2024-05-21T17:41:40.648 INFO:teuthology.orchestra.run.smithi003.stdout:38654705670 2024-05-21T17:41:40.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 10 mgrc shutdown 2024-05-21T17:41:40.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 -- 172.21.15.3:0/858042564 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f18200af1e0 msgr2=0x7f18200b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f18200af1e0 0x7f18200b16a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1838000f60 tx=0x7f1838009040 comp rx=0 tx=0).stop 2024-05-21T17:41:40.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 -- 172.21.15.3:0/858042564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c147130 msgr2=0x7f183c09cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c147130 0x7f183c09cad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f183400c6b0 tx=0x7f183400cb70 comp rx=0 tx=0).stop 2024-05-21T17:41:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 -- 172.21.15.3:0/858042564 shutdown_connections 2024-05-21T17:41:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f18200af1e0 0x7f18200b16a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f183c147130 0x7f183c09cad0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f183c0ac530 0x7f183c09c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 --2- 172.21.15.3:0/858042564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f183c0aa2f0 0x7f183c09dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 -- 172.21.15.3:0/858042564 >> 172.21.15.3:0/858042564 conn(0x7f183c0bba40 msgr2=0x7f183c0b98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 -- 172.21.15.3:0/858042564 shutdown_connections 2024-05-21T17:41:40.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.649+0000 7f1844283640 1 -- 172.21.15.3:0/858042564 wait complete. 2024-05-21T17:41:40.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.1 2024-05-21T17:41:40.662 DEBUG:teuthology.parallel:result is None 2024-05-21T17:41:40.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.682+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb6ac073c20 con 0x7fb6ac146890 2024-05-21T17:41:40.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.683+0000 7fbff44f5640 1 -- 172.21.15.3:0/1166685580 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbfec072200 con 0x7fbfec06dce0 2024-05-21T17:41:40.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.683+0000 7fb6a37fe640 1 -- 172.21.15.3:0/2519315701 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb694040070 con 0x7fb6ac146890 2024-05-21T17:41:40.684 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2024-05-21T17:41:40.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.683+0000 7fbfe2ffd640 1 -- 172.21.15.3:0/1166685580 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbfe80cabc0 con 0x7fbfec06dce0 2024-05-21T17:41:40.685 INFO:teuthology.orchestra.run.smithi003.stdout:38654705669 2024-05-21T17:41:40.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 10 mgrc shutdown 2024-05-21T17:41:40.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb68c0af210 msgr2=0x7fb68c0b16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb68c0af210 0x7fb68c0b16d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb69c003bc0 tx=0x7fb69c009970 comp rx=0 tx=0).stop 2024-05-21T17:41:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 msgr2=0x7fb6ac09ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac09ef80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb694033000 tx=0x7fb6940046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 shutdown_connections 2024-05-21T17:41:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb68c0af210 0x7fb68c0b16d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6ac149850 0x7fb6ac1403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb6ac146890 0x7fb6ac09ef80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 --2- 172.21.15.3:0/2519315701 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb6ac0a6f50 0x7fb6ac0b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 >> 172.21.15.3:0/2519315701 conn(0x7fb6ac0a8c40 msgr2=0x7fb6ac0aa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 shutdown_connections 2024-05-21T17:41:40.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.685+0000 7fb6b11b8640 1 -- 172.21.15.3:0/2519315701 wait complete. 2024-05-21T17:41:40.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 10 mgrc shutdown 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 -- 172.21.15.3:0/1166685580 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbfd00af260 msgr2=0x7fbfd00b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbfd00af260 0x7fbfd00b1720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbfec135e20 tx=0x7fbfdc008040 comp rx=0 tx=0).stop 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 -- 172.21.15.3:0/1166685580 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec06dce0 msgr2=0x7fbfec0ad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec06dce0 0x7fbfec0ad7b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbfe802e090 tx=0x7fbfe80046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 -- 172.21.15.3:0/1166685580 shutdown_connections 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fbfd00af260 0x7fbfd00b1720 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfec132820 0x7fbfec1cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfec073b20 0x7fbfec1ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 --2- 172.21.15.3:0/1166685580 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fbfec06dce0 0x7fbfec0ad7b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 -- 172.21.15.3:0/1166685580 >> 172.21.15.3:0/1166685580 conn(0x7fbfec0713e0 msgr2=0x7fbfec06c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 -- 172.21.15.3:0/1166685580 shutdown_connections 2024-05-21T17:41:40.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.686+0000 7fbff44f5640 1 -- 172.21.15.3:0/1166685580 wait complete. 2024-05-21T17:41:40.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2024-05-21T17:41:40.698 DEBUG:teuthology.parallel:result is None 2024-05-21T17:41:40.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.3 2024-05-21T17:41:40.703 DEBUG:teuthology.parallel:result is None 2024-05-21T17:41:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.753+0000 7f2db56f5640 1 -- 172.21.15.3:0/1187218647 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2db0069290 con 0x7f2db01327b0 2024-05-21T17:41:40.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.753+0000 7f2d977fe640 1 -- 172.21.15.3:0/1187218647 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2da004b030 con 0x7f2db01327b0 2024-05-21T17:41:40.755 INFO:teuthology.orchestra.run.smithi003.stdout:38654705670 2024-05-21T17:41:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 10 mgrc shutdown 2024-05-21T17:41:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 -- 172.21.15.3:0/1187218647 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f2d9c0af260 msgr2=0x7f2d9c0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f2d9c0af260 0x7f2d9c0b1720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009560 tx=0x7f2da4008040 comp rx=0 tx=0).stop 2024-05-21T17:41:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 -- 172.21.15.3:0/1187218647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 msgr2=0x7f2db0134d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db0134d50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2da002b780 tx=0x7f2da00046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 -- 172.21.15.3:0/1187218647 shutdown_connections 2024-05-21T17:41:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f2d9c0af260 0x7f2d9c0b1720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2db0139470 0x7f2db01427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2db0132b90 0x7f2db00b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 --2- 172.21.15.3:0/1187218647 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f2db01327b0 0x7f2db0134d50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 -- 172.21.15.3:0/1187218647 >> 172.21.15.3:0/1187218647 conn(0x7f2db00713c0 msgr2=0x7f2db006d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 -- 172.21.15.3:0/1187218647 shutdown_connections 2024-05-21T17:41:40.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.755+0000 7f2db56f5640 1 -- 172.21.15.3:0/1187218647 wait complete. 2024-05-21T17:41:40.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.2 2024-05-21T17:41:40.767 DEBUG:teuthology.parallel:result is None 2024-05-21T17:41:40.767 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-21T17:41:40.768 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T17:41:40.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.869+0000 7fb9239bf640 1 Processor -- start 2024-05-21T17:41:40.871 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.870+0000 7fb9239bf640 1 -- start start 2024-05-21T17:41:40.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.870+0000 7fb9239bf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb91c145ba0 0x7fb91c0ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.870+0000 7fb9239bf640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c06c770 0x7fb91c145f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.870+0000 7fb9239bf640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb91c146990 con 0x7fb91c06cb50 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.870+0000 7fb9239bf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fb91c146c30 con 0x7fb91c145ba0 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.870+0000 7fb9239bf640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fb91c06d870 con 0x7fb91c06c770 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.870+0000 7fb921f5c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fb91c06cb50 0x7fb91c149650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60726/0 (socket says 172.21.15.3:60726) 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.870+0000 7fb921f5c640 1 -- 172.21.15.3:0/2947926603 learned_addr learned my addr 172.21.15.3:0/2947926603 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.870+0000 7fb920f5a640 1 --2- 172.21.15.3:0/2947926603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c06c770 0x7fb91c145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.871+0000 7fb913fff640 1 -- 172.21.15.3:0/2947926603 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2938215299 0 0) 0x7fb91c146990 con 0x7fb91c06cb50 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.871+0000 7fb92175b640 1 --2- 172.21.15.3:0/2947926603 >> v2:172.21.15.203:3300/0 conn(0x7fb91c145ba0 0x7fb91c0ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.871+0000 7fb913fff640 1 -- 172.21.15.3:0/2947926603 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb908003520 con 0x7fb91c06cb50 2024-05-21T17:41:40.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.871+0000 7fb920f5a640 1 -- 172.21.15.3:0/2947926603 >> v2:172.21.15.203:3300/0 conn(0x7fb91c145ba0 msgr2=0x7fb91c0ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:40.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.871+0000 7fb920f5a640 1 --2- 172.21.15.3:0/2947926603 >> v2:172.21.15.203:3300/0 conn(0x7fb91c145ba0 0x7fb91c0ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.871+0000 7fb920f5a640 1 -- 172.21.15.3:0/2947926603 >> v1:172.21.15.3:6789/0 conn(0x7fb91c06cb50 legacy=0x7fb91c149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.871+0000 7fb920f5a640 1 -- 172.21.15.3:0/2947926603 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c0adc80 con 0x7fb91c06c770 2024-05-21T17:41:40.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.871+0000 7fb92175b640 1 --2- 172.21.15.3:0/2947926603 >> v2:172.21.15.203:3300/0 conn(0x7fb91c145ba0 0x7fb91c0ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:40.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb920f5a640 1 --2- 172.21.15.3:0/2947926603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c06c770 0x7fb91c145f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0097d0 tx=0x7fb90c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dad40453bfa0ea server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb913fff640 1 -- 172.21.15.3:0/2947926603 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c035020 con 0x7fb91c06c770 2024-05-21T17:41:40.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb913fff640 1 -- 172.21.15.3:0/2947926603 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c031890 con 0x7fb91c06c770 2024-05-21T17:41:40.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb913fff640 1 -- 172.21.15.3:0/2947926603 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c031c60 con 0x7fb91c06c770 2024-05-21T17:41:40.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb9239bf640 1 -- 172.21.15.3:0/2947926603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c06c770 msgr2=0x7fb91c145f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb9239bf640 1 --2- 172.21.15.3:0/2947926603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c06c770 0x7fb91c145f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0097d0 tx=0x7fb90c02b930 comp rx=0 tx=0).stop 2024-05-21T17:41:40.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb9239bf640 1 -- 172.21.15.3:0/2947926603 shutdown_connections 2024-05-21T17:41:40.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb9239bf640 1 --2- 172.21.15.3:0/2947926603 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c06c770 0x7fb91c145f80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb9239bf640 1 --2- 172.21.15.3:0/2947926603 >> v2:172.21.15.203:3300/0 conn(0x7fb91c145ba0 0x7fb91c0ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb9239bf640 1 -- 172.21.15.3:0/2947926603 >> 172.21.15.3:0/2947926603 conn(0x7fb91c0bbc20 msgr2=0x7fb91c0bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:40.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb9239bf640 1 -- 172.21.15.3:0/2947926603 shutdown_connections 2024-05-21T17:41:40.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.872+0000 7fb9239bf640 1 -- 172.21.15.3:0/2947926603 wait complete. 2024-05-21T17:41:40.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.873+0000 7fb9239bf640 1 Processor -- start 2024-05-21T17:41:40.877 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb9239bf640 1 -- start start 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb9239bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb91c06c770 0x7fb91c0a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb9239bf640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c145ba0 0x7fb91c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb9239bf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb91c0af5e0 0x7fb91c1def50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb9239bf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c1464c0 con 0x7fb91c0af5e0 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb9239bf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c0bc920 con 0x7fb91c06c770 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb9239bf640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fb91c146640 con 0x7fb91c145ba0 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb920f5a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c145ba0 0x7fb91c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb920f5a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c145ba0 0x7fb91c0b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41094/0 (socket says 172.21.15.3:41094) 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb920f5a640 1 -- 172.21.15.3:0/811507339 learned_addr learned my addr 172.21.15.3:0/811507339 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb921f5c640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb91c0af5e0 0x7fb91c1def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb92175b640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb91c06c770 0x7fb91c0a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb920f5a640 1 -- 172.21.15.3:0/811507339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb91c06c770 msgr2=0x7fb91c0a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb920f5a640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb91c06c770 0x7fb91c0a7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.874+0000 7fb920f5a640 1 -- 172.21.15.3:0/811507339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb91c0af5e0 msgr2=0x7fb91c1def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.875+0000 7fb920f5a640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb91c0af5e0 0x7fb91c1def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.875+0000 7fb920f5a640 1 -- 172.21.15.3:0/811507339 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c1d08a0 con 0x7fb91c145ba0 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.875+0000 7fb921f5c640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb91c0af5e0 0x7fb91c1def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.875+0000 7fb920f5a640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c145ba0 0x7fb91c0b0530 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb90c02bf40 tx=0x7fb90c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.875+0000 7fb911ffb640 1 -- 172.21.15.3:0/811507339 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c035040 con 0x7fb91c145ba0 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.875+0000 7fb911ffb640 1 -- 172.21.15.3:0/811507339 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90c030d20 con 0x7fb91c145ba0 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.875+0000 7fb9239bf640 1 -- 172.21.15.3:0/811507339 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91c0afd50 con 0x7fb91c145ba0 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.875+0000 7fb911ffb640 1 -- 172.21.15.3:0/811507339 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0466a0 con 0x7fb91c145ba0 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.875+0000 7fb9239bf640 1 -- 172.21.15.3:0/811507339 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91c142ec0 con 0x7fb91c145ba0 2024-05-21T17:41:40.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.876+0000 7fb9239bf640 1 -- 172.21.15.3:0/811507339 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91c069a90 con 0x7fb91c145ba0 2024-05-21T17:41:40.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.878+0000 7fb911ffb640 1 -- 172.21.15.3:0/811507339 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7fb90c038020 con 0x7fb91c145ba0 2024-05-21T17:41:40.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.878+0000 7fb911ffb640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:40.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.879+0000 7fb911ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:40.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.879+0000 7fb911ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:40.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.879+0000 7fb911ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:40.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.879+0000 7fb911ffb640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb9000af260 0x7fb9000b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:40.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.879+0000 7fb911ffb640 1 -- 172.21.15.3:0/811507339 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7fb90c0ff3a0 con 0x7fb91c145ba0 2024-05-21T17:41:40.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.879+0000 7fb92175b640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb9000af260 0x7fb9000b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:40.881 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.880+0000 7fb92175b640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb9000af260 0x7fb9000b1720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb918008ee0 tx=0x7fb918008430 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:40.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:40.882+0000 7fb911ffb640 1 -- 172.21.15.3:0/811507339 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb90c02d040 con 0x7fb91c145ba0 2024-05-21T17:41:41.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.070+0000 7fb9239bf640 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2024-05-21T17:41:41.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.070+0000 7fb9239bf640 1 -- 172.21.15.3:0/811507339 --> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb91c06d380 con 0x7fb9000af260 2024-05-21T17:41:41.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.077+0000 7fb911ffb640 1 -- 172.21.15.3:0/811507339 <== mgr.4102 v2:172.21.15.203:6824/4157202067 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+268601 (secure 0 0 0) 0x7fb91c06d380 con 0x7fb9000af260 2024-05-21T17:41:41.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.077+0000 7fb911ffb640 20 mgrc handle_command_reply tid 0 r 0 2024-05-21T17:41:41.081 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:41.082 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-21T17:41:41.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 10 mgrc shutdown 2024-05-21T17:41:41.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 1 -- 172.21.15.3:0/811507339 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb9000af260 msgr2=0x7fb9000b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb9000af260 0x7fb9000b1720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb918008ee0 tx=0x7fb918008430 comp rx=0 tx=0).stop 2024-05-21T17:41:41.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 1 -- 172.21.15.3:0/811507339 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c145ba0 msgr2=0x7fb91c0b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c145ba0 0x7fb91c0b0530 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb90c02bf40 tx=0x7fb90c0046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 1 -- 172.21.15.3:0/811507339 shutdown_connections 2024-05-21T17:41:41.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fb9000af260 0x7fb9000b1720 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb91c0af5e0 0x7fb91c1def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fb91c145ba0 0x7fb91c0b0530 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 1 --2- 172.21.15.3:0/811507339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb91c06c770 0x7fb91c0a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.080+0000 7fb9239bf640 1 -- 172.21.15.3:0/811507339 >> 172.21.15.3:0/811507339 conn(0x7fb91c0bbc20 msgr2=0x7fb91c0bb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:41.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.081+0000 7fb9239bf640 1 -- 172.21.15.3:0/811507339 shutdown_connections 2024-05-21T17:41:41.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.081+0000 7fb9239bf640 1 -- 172.21.15.3:0/811507339 wait complete. 2024-05-21T17:41:41.092 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-21T17:41:39.519719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":197,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108856,"kb_used_data":768,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377378504,"statfs":{"total":386547056640,"available":386435588096,"internally_reserved":0,"allocated":786432,"data_stored":1388172,"data_compressed":17264,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":6377,"internal_metadata":110618391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.351933"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181633+0000","last_change":"2024-05-21T17:41:33.197173+0000","last_active":"2024-05-21T17:41:35.181633+0000","last_peered":"2024-05-21T17:41:35.181633+0000","last_clean":"2024-05-21T17:41:35.181633+0000","last_became_active":"2024-05-21T17:41:33.196781+0000","last_became_peered":"2024-05-21T17:41:33.196781+0000","last_unstale":"2024-05-21T17:41:35.181633+0000","last_undegraded":"2024-05-21T17:41:35.181633+0000","last_fullsized":"2024-05-21T17:41:35.181633+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:12:57.454391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181522+0000","last_change":"2024-05-21T17:41:33.197965+0000","last_active":"2024-05-21T17:41:35.181522+0000","last_peered":"2024-05-21T17:41:35.181522+0000","last_clean":"2024-05-21T17:41:35.181522+0000","last_became_active":"2024-05-21T17:41:33.197693+0000","last_became_peered":"2024-05-21T17:41:33.197693+0000","last_unstale":"2024-05-21T17:41:35.181522+0000","last_undegraded":"2024-05-21T17:41:35.181522+0000","last_fullsized":"2024-05-21T17:41:35.181522+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:40:32.331027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504514+0000","last_change":"2024-05-21T17:41:33.212529+0000","last_active":"2024-05-21T17:41:35.504514+0000","last_peered":"2024-05-21T17:41:35.504514+0000","last_clean":"2024-05-21T17:41:35.504514+0000","last_became_active":"2024-05-21T17:41:33.211026+0000","last_became_peered":"2024-05-21T17:41:33.211026+0000","last_unstale":"2024-05-21T17:41:35.504514+0000","last_undegraded":"2024-05-21T17:41:35.504514+0000","last_fullsized":"2024-05-21T17:41:35.504514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:15:01.504097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196450+0000","last_change":"2024-05-21T17:41:33.191234+0000","last_active":"2024-05-21T17:41:34.196450+0000","last_peered":"2024-05-21T17:41:34.196450+0000","last_clean":"2024-05-21T17:41:34.196450+0000","last_became_active":"2024-05-21T17:41:33.190138+0000","last_became_peered":"2024-05-21T17:41:33.190138+0000","last_unstale":"2024-05-21T17:41:34.196450+0000","last_undegraded":"2024-05-21T17:41:34.196450+0000","last_fullsized":"2024-05-21T17:41:34.196450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:31.958772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504393+0000","last_change":"2024-05-21T17:41:33.203349+0000","last_active":"2024-05-21T17:41:35.504393+0000","last_peered":"2024-05-21T17:41:35.504393+0000","last_clean":"2024-05-21T17:41:35.504393+0000","last_became_active":"2024-05-21T17:41:33.203059+0000","last_became_peered":"2024-05-21T17:41:33.203059+0000","last_unstale":"2024-05-21T17:41:35.504393+0000","last_undegraded":"2024-05-21T17:41:35.504393+0000","last_fullsized":"2024-05-21T17:41:35.504393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:10:26.347216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504144+0000","last_change":"2024-05-21T17:41:33.208230+0000","last_active":"2024-05-21T17:41:35.504144+0000","last_peered":"2024-05-21T17:41:35.504144+0000","last_clean":"2024-05-21T17:41:35.504144+0000","last_became_active":"2024-05-21T17:41:33.207955+0000","last_became_peered":"2024-05-21T17:41:33.207955+0000","last_unstale":"2024-05-21T17:41:35.504144+0000","last_undegraded":"2024-05-21T17:41:35.504144+0000","last_fullsized":"2024-05-21T17:41:35.504144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:42:25.073971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503817+0000","last_change":"2024-05-21T17:41:33.208443+0000","last_active":"2024-05-21T17:41:35.503817+0000","last_peered":"2024-05-21T17:41:35.503817+0000","last_clean":"2024-05-21T17:41:35.503817+0000","last_became_active":"2024-05-21T17:41:33.208038+0000","last_became_peered":"2024-05-21T17:41:33.208038+0000","last_unstale":"2024-05-21T17:41:35.503817+0000","last_undegraded":"2024-05-21T17:41:35.503817+0000","last_fullsized":"2024-05-21T17:41:35.503817+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:14:16.199161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195697+0000","last_change":"2024-05-21T17:41:33.202443+0000","last_active":"2024-05-21T17:41:34.195697+0000","last_peered":"2024-05-21T17:41:34.195697+0000","last_clean":"2024-05-21T17:41:34.195697+0000","last_became_active":"2024-05-21T17:41:33.200436+0000","last_became_peered":"2024-05-21T17:41:33.200436+0000","last_unstale":"2024-05-21T17:41:34.195697+0000","last_undegraded":"2024-05-21T17:41:34.195697+0000","last_fullsized":"2024-05-21T17:41:34.195697+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:41:08.380018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197981+0000","last_change":"2024-05-21T17:41:33.185540+0000","last_active":"2024-05-21T17:41:34.197981+0000","last_peered":"2024-05-21T17:41:34.197981+0000","last_clean":"2024-05-21T17:41:34.197981+0000","last_became_active":"2024-05-21T17:41:33.185269+0000","last_became_peered":"2024-05-21T17:41:33.185269+0000","last_unstale":"2024-05-21T17:41:34.197981+0000","last_undegraded":"2024-05-21T17:41:34.197981+0000","last_fullsized":"2024-05-21T17:41:34.197981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:29:07.891173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196019+0000","last_change":"2024-05-21T17:41:33.204222+0000","last_active":"2024-05-21T17:41:34.196019+0000","last_peered":"2024-05-21T17:41:34.196019+0000","last_clean":"2024-05-21T17:41:34.196019+0000","last_became_active":"2024-05-21T17:41:33.203828+0000","last_became_peered":"2024-05-21T17:41:33.203828+0000","last_unstale":"2024-05-21T17:41:34.196019+0000","last_undegraded":"2024-05-21T17:41:34.196019+0000","last_fullsized":"2024-05-21T17:41:34.196019+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:09.846517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196360+0000","last_change":"2024-05-21T17:41:33.190185+0000","last_active":"2024-05-21T17:41:34.196360+0000","last_peered":"2024-05-21T17:41:34.196360+0000","last_clean":"2024-05-21T17:41:34.196360+0000","last_became_active":"2024-05-21T17:41:33.189865+0000","last_became_peered":"2024-05-21T17:41:33.189865+0000","last_unstale":"2024-05-21T17:41:34.196360+0000","last_undegraded":"2024-05-21T17:41:34.196360+0000","last_fullsized":"2024-05-21T17:41:34.196360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:36:50.231113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181658+0000","last_change":"2024-05-21T17:41:33.212714+0000","last_active":"2024-05-21T17:41:35.181658+0000","last_peered":"2024-05-21T17:41:35.181658+0000","last_clean":"2024-05-21T17:41:35.181658+0000","last_became_active":"2024-05-21T17:41:33.212447+0000","last_became_peered":"2024-05-21T17:41:33.212447+0000","last_unstale":"2024-05-21T17:41:35.181658+0000","last_undegraded":"2024-05-21T17:41:35.181658+0000","last_fullsized":"2024-05-21T17:41:35.181658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:06.343413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197751+0000","last_change":"2024-05-21T17:41:33.194642+0000","last_active":"2024-05-21T17:41:34.197751+0000","last_peered":"2024-05-21T17:41:34.197751+0000","last_clean":"2024-05-21T17:41:34.197751+0000","last_became_active":"2024-05-21T17:41:33.194331+0000","last_became_peered":"2024-05-21T17:41:33.194331+0000","last_unstale":"2024-05-21T17:41:34.197751+0000","last_undegraded":"2024-05-21T17:41:34.197751+0000","last_fullsized":"2024-05-21T17:41:34.197751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:54:27.905968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195573+0000","last_change":"2024-05-21T17:41:33.200546+0000","last_active":"2024-05-21T17:41:34.195573+0000","last_peered":"2024-05-21T17:41:34.195573+0000","last_clean":"2024-05-21T17:41:34.195573+0000","last_became_active":"2024-05-21T17:41:33.199969+0000","last_became_peered":"2024-05-21T17:41:33.199969+0000","last_unstale":"2024-05-21T17:41:34.195573+0000","last_undegraded":"2024-05-21T17:41:34.195573+0000","last_fullsized":"2024-05-21T17:41:34.195573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:06.343413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195553+0000","last_change":"2024-05-21T17:41:33.200283+0000","last_active":"2024-05-21T17:41:34.195553+0000","last_peered":"2024-05-21T17:41:34.195553+0000","last_clean":"2024-05-21T17:41:34.195553+0000","last_became_active":"2024-05-21T17:41:33.199931+0000","last_became_peered":"2024-05-21T17:41:33.199931+0000","last_unstale":"2024-05-21T17:41:34.195553+0000","last_undegraded":"2024-05-21T17:41:34.195553+0000","last_fullsized":"2024-05-21T17:41:34.195553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:12:57.454391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503627+0000","last_change":"2024-05-21T17:41:33.208399+0000","last_active":"2024-05-21T17:41:35.503627+0000","last_peered":"2024-05-21T17:41:35.503627+0000","last_clean":"2024-05-21T17:41:35.503627+0000","last_became_active":"2024-05-21T17:41:33.208104+0000","last_became_peered":"2024-05-21T17:41:33.208104+0000","last_unstale":"2024-05-21T17:41:35.503627+0000","last_undegraded":"2024-05-21T17:41:35.503627+0000","last_fullsized":"2024-05-21T17:41:35.503627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:14:30.272478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196463+0000","last_change":"2024-05-21T17:41:33.200479+0000","last_active":"2024-05-21T17:41:34.196463+0000","last_peered":"2024-05-21T17:41:34.196463+0000","last_clean":"2024-05-21T17:41:34.196463+0000","last_became_active":"2024-05-21T17:41:33.199926+0000","last_became_peered":"2024-05-21T17:41:33.199926+0000","last_unstale":"2024-05-21T17:41:34.196463+0000","last_undegraded":"2024-05-21T17:41:34.196463+0000","last_fullsized":"2024-05-21T17:41:34.196463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:03:55.087212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504684+0000","last_change":"2024-05-21T17:41:33.208923+0000","last_active":"2024-05-21T17:41:35.504684+0000","last_peered":"2024-05-21T17:41:35.504684+0000","last_clean":"2024-05-21T17:41:35.504684+0000","last_became_active":"2024-05-21T17:41:33.208377+0000","last_became_peered":"2024-05-21T17:41:33.208377+0000","last_unstale":"2024-05-21T17:41:35.504684+0000","last_undegraded":"2024-05-21T17:41:35.504684+0000","last_fullsized":"2024-05-21T17:41:35.504684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:49.839600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197601+0000","last_change":"2024-05-21T17:41:33.214811+0000","last_active":"2024-05-21T17:41:34.197601+0000","last_peered":"2024-05-21T17:41:34.197601+0000","last_clean":"2024-05-21T17:41:34.197601+0000","last_became_active":"2024-05-21T17:41:33.214502+0000","last_became_peered":"2024-05-21T17:41:33.214502+0000","last_unstale":"2024-05-21T17:41:34.197601+0000","last_undegraded":"2024-05-21T17:41:34.197601+0000","last_fullsized":"2024-05-21T17:41:34.197601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:11:24.435803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196070+0000","last_change":"2024-05-21T17:41:33.190432+0000","last_active":"2024-05-21T17:41:34.196070+0000","last_peered":"2024-05-21T17:41:34.196070+0000","last_clean":"2024-05-21T17:41:34.196070+0000","last_became_active":"2024-05-21T17:41:33.189727+0000","last_became_peered":"2024-05-21T17:41:33.189727+0000","last_unstale":"2024-05-21T17:41:34.196070+0000","last_undegraded":"2024-05-21T17:41:34.196070+0000","last_fullsized":"2024-05-21T17:41:34.196070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:45:43.258694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195941+0000","last_change":"2024-05-21T17:41:33.206591+0000","last_active":"2024-05-21T17:41:34.195941+0000","last_peered":"2024-05-21T17:41:34.195941+0000","last_clean":"2024-05-21T17:41:34.195941+0000","last_became_active":"2024-05-21T17:41:33.205156+0000","last_became_peered":"2024-05-21T17:41:33.205156+0000","last_unstale":"2024-05-21T17:41:34.195941+0000","last_undegraded":"2024-05-21T17:41:34.195941+0000","last_fullsized":"2024-05-21T17:41:34.195941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:55:26.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195772+0000","last_change":"2024-05-21T17:41:33.200833+0000","last_active":"2024-05-21T17:41:34.195772+0000","last_peered":"2024-05-21T17:41:34.195772+0000","last_clean":"2024-05-21T17:41:34.195772+0000","last_became_active":"2024-05-21T17:41:33.200208+0000","last_became_peered":"2024-05-21T17:41:33.200208+0000","last_unstale":"2024-05-21T17:41:34.195772+0000","last_undegraded":"2024-05-21T17:41:34.195772+0000","last_fullsized":"2024-05-21T17:41:34.195772+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:28:20.455810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503916+0000","last_change":"2024-05-21T17:41:33.214524+0000","last_active":"2024-05-21T17:41:35.503916+0000","last_peered":"2024-05-21T17:41:35.503916+0000","last_clean":"2024-05-21T17:41:35.503916+0000","last_became_active":"2024-05-21T17:41:33.211351+0000","last_became_peered":"2024-05-21T17:41:33.211351+0000","last_unstale":"2024-05-21T17:41:35.503916+0000","last_undegraded":"2024-05-21T17:41:35.503916+0000","last_fullsized":"2024-05-21T17:41:35.503916+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:25:58.491664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195655+0000","last_change":"2024-05-21T17:41:33.194325+0000","last_active":"2024-05-21T17:41:34.195655+0000","last_peered":"2024-05-21T17:41:34.195655+0000","last_clean":"2024-05-21T17:41:34.195655+0000","last_became_active":"2024-05-21T17:41:33.193802+0000","last_became_peered":"2024-05-21T17:41:33.193802+0000","last_unstale":"2024-05-21T17:41:34.195655+0000","last_undegraded":"2024-05-21T17:41:34.195655+0000","last_fullsized":"2024-05-21T17:41:34.195655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:38:18.545593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196458+0000","last_change":"2024-05-21T17:41:33.209308+0000","last_active":"2024-05-21T17:41:34.196458+0000","last_peered":"2024-05-21T17:41:34.196458+0000","last_clean":"2024-05-21T17:41:34.196458+0000","last_became_active":"2024-05-21T17:41:33.209014+0000","last_became_peered":"2024-05-21T17:41:33.209014+0000","last_unstale":"2024-05-21T17:41:34.196458+0000","last_undegraded":"2024-05-21T17:41:34.196458+0000","last_fullsized":"2024-05-21T17:41:34.196458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:14:30.272478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197581+0000","last_change":"2024-05-21T17:41:33.197441+0000","last_active":"2024-05-21T17:41:34.197581+0000","last_peered":"2024-05-21T17:41:34.197581+0000","last_clean":"2024-05-21T17:41:34.197581+0000","last_became_active":"2024-05-21T17:41:33.196890+0000","last_became_peered":"2024-05-21T17:41:33.196890+0000","last_unstale":"2024-05-21T17:41:34.197581+0000","last_undegraded":"2024-05-21T17:41:34.197581+0000","last_fullsized":"2024-05-21T17:41:34.197581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:29:26.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505181+0000","last_change":"2024-05-21T17:41:33.214392+0000","last_active":"2024-05-21T17:41:35.505181+0000","last_peered":"2024-05-21T17:41:35.505181+0000","last_clean":"2024-05-21T17:41:35.505181+0000","last_became_active":"2024-05-21T17:41:33.213788+0000","last_became_peered":"2024-05-21T17:41:33.213788+0000","last_unstale":"2024-05-21T17:41:35.505181+0000","last_undegraded":"2024-05-21T17:41:35.505181+0000","last_fullsized":"2024-05-21T17:41:35.505181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:33:54.977766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198327+0000","last_change":"2024-05-21T17:41:33.197602+0000","last_active":"2024-05-21T17:41:34.198327+0000","last_peered":"2024-05-21T17:41:34.198327+0000","last_clean":"2024-05-21T17:41:34.198327+0000","last_became_active":"2024-05-21T17:41:33.196498+0000","last_became_peered":"2024-05-21T17:41:33.196498+0000","last_unstale":"2024-05-21T17:41:34.198327+0000","last_undegraded":"2024-05-21T17:41:34.198327+0000","last_fullsized":"2024-05-21T17:41:34.198327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:06.343413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197904+0000","last_change":"2024-05-21T17:41:33.194996+0000","last_active":"2024-05-21T17:41:34.197904+0000","last_peered":"2024-05-21T17:41:34.197904+0000","last_clean":"2024-05-21T17:41:34.197904+0000","last_became_active":"2024-05-21T17:41:33.194676+0000","last_became_peered":"2024-05-21T17:41:33.194676+0000","last_unstale":"2024-05-21T17:41:34.197904+0000","last_undegraded":"2024-05-21T17:41:34.197904+0000","last_fullsized":"2024-05-21T17:41:34.197904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:48.164464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196316+0000","last_change":"2024-05-21T17:41:33.209301+0000","last_active":"2024-05-21T17:41:34.196316+0000","last_peered":"2024-05-21T17:41:34.196316+0000","last_clean":"2024-05-21T17:41:34.196316+0000","last_became_active":"2024-05-21T17:41:33.209030+0000","last_became_peered":"2024-05-21T17:41:33.209030+0000","last_unstale":"2024-05-21T17:41:34.196316+0000","last_undegraded":"2024-05-21T17:41:34.196316+0000","last_fullsized":"2024-05-21T17:41:34.196316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:46:10.096945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197268+0000","last_change":"2024-05-21T17:41:33.185555+0000","last_active":"2024-05-21T17:41:34.197268+0000","last_peered":"2024-05-21T17:41:34.197268+0000","last_clean":"2024-05-21T17:41:34.197268+0000","last_became_active":"2024-05-21T17:41:33.185206+0000","last_became_peered":"2024-05-21T17:41:33.185206+0000","last_unstale":"2024-05-21T17:41:34.197268+0000","last_undegraded":"2024-05-21T17:41:34.197268+0000","last_fullsized":"2024-05-21T17:41:34.197268+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:53:00.383833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196388+0000","last_change":"2024-05-21T17:41:33.194078+0000","last_active":"2024-05-21T17:41:34.196388+0000","last_peered":"2024-05-21T17:41:34.196388+0000","last_clean":"2024-05-21T17:41:34.196388+0000","last_became_active":"2024-05-21T17:41:33.190990+0000","last_became_peered":"2024-05-21T17:41:33.190990+0000","last_unstale":"2024-05-21T17:41:34.196388+0000","last_undegraded":"2024-05-21T17:41:34.196388+0000","last_fullsized":"2024-05-21T17:41:34.196388+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:23:29.692427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504416+0000","last_change":"2024-05-21T17:41:33.203400+0000","last_active":"2024-05-21T17:41:35.504416+0000","last_peered":"2024-05-21T17:41:35.504416+0000","last_clean":"2024-05-21T17:41:35.504416+0000","last_became_active":"2024-05-21T17:41:33.203117+0000","last_became_peered":"2024-05-21T17:41:33.203117+0000","last_unstale":"2024-05-21T17:41:35.504416+0000","last_undegraded":"2024-05-21T17:41:35.504416+0000","last_fullsized":"2024-05-21T17:41:35.504416+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:30:00.800497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504541+0000","last_change":"2024-05-21T17:41:32.194821+0000","last_active":"2024-05-21T17:41:35.504541+0000","last_peered":"2024-05-21T17:41:35.504541+0000","last_clean":"2024-05-21T17:41:35.504541+0000","last_became_active":"2024-05-21T17:41:32.194501+0000","last_became_peered":"2024-05-21T17:41:32.194501+0000","last_unstale":"2024-05-21T17:41:35.504541+0000","last_undegraded":"2024-05-21T17:41:35.504541+0000","last_fullsized":"2024-05-21T17:41:35.504541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:32:31.739397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505001+0000","last_change":"2024-05-21T17:41:33.206606+0000","last_active":"2024-05-21T17:41:35.505001+0000","last_peered":"2024-05-21T17:41:35.505001+0000","last_clean":"2024-05-21T17:41:35.505001+0000","last_became_active":"2024-05-21T17:41:33.206194+0000","last_became_peered":"2024-05-21T17:41:33.206194+0000","last_unstale":"2024-05-21T17:41:35.505001+0000","last_undegraded":"2024-05-21T17:41:35.505001+0000","last_fullsized":"2024-05-21T17:41:35.505001+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:04:01.510328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181288+0000","last_change":"2024-05-21T17:41:32.183395+0000","last_active":"2024-05-21T17:41:35.181288+0000","last_peered":"2024-05-21T17:41:35.181288+0000","last_clean":"2024-05-21T17:41:35.181288+0000","last_became_active":"2024-05-21T17:41:32.183108+0000","last_became_peered":"2024-05-21T17:41:32.183108+0000","last_unstale":"2024-05-21T17:41:35.181288+0000","last_undegraded":"2024-05-21T17:41:35.181288+0000","last_fullsized":"2024-05-21T17:41:35.181288+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:33.195952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504687+0000","last_change":"2024-05-21T17:41:33.198913+0000","last_active":"2024-05-21T17:41:35.504687+0000","last_peered":"2024-05-21T17:41:35.504687+0000","last_clean":"2024-05-21T17:41:35.504687+0000","last_became_active":"2024-05-21T17:41:33.198607+0000","last_became_peered":"2024-05-21T17:41:33.198607+0000","last_unstale":"2024-05-21T17:41:35.504687+0000","last_undegraded":"2024-05-21T17:41:35.504687+0000","last_fullsized":"2024-05-21T17:41:35.504687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:14:01.670496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504299+0000","last_change":"2024-05-21T17:41:32.194461+0000","last_active":"2024-05-21T17:41:35.504299+0000","last_peered":"2024-05-21T17:41:35.504299+0000","last_clean":"2024-05-21T17:41:35.504299+0000","last_became_active":"2024-05-21T17:41:32.194194+0000","last_became_peered":"2024-05-21T17:41:32.194194+0000","last_unstale":"2024-05-21T17:41:35.504299+0000","last_undegraded":"2024-05-21T17:41:35.504299+0000","last_fullsized":"2024-05-21T17:41:35.504299+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:00:15.399735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198049+0000","last_change":"2024-05-21T17:41:33.198965+0000","last_active":"2024-05-21T17:41:34.198049+0000","last_peered":"2024-05-21T17:41:34.198049+0000","last_clean":"2024-05-21T17:41:34.198049+0000","last_became_active":"2024-05-21T17:41:33.197958+0000","last_became_peered":"2024-05-21T17:41:33.197958+0000","last_unstale":"2024-05-21T17:41:34.198049+0000","last_undegraded":"2024-05-21T17:41:34.198049+0000","last_fullsized":"2024-05-21T17:41:34.198049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:06:55.657012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"15'8","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196258+0000","last_change":"2024-05-21T17:41:32.194159+0000","last_active":"2024-05-21T17:41:34.196258+0000","last_peered":"2024-05-21T17:41:34.196258+0000","last_clean":"2024-05-21T17:41:34.196258+0000","last_became_active":"2024-05-21T17:41:32.193896+0000","last_became_peered":"2024-05-21T17:41:32.193896+0000","last_unstale":"2024-05-21T17:41:34.196258+0000","last_undegraded":"2024-05-21T17:41:34.196258+0000","last_fullsized":"2024-05-21T17:41:34.196258+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:31:47.729777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181569+0000","last_change":"2024-05-21T17:41:32.186754+0000","last_active":"2024-05-21T17:41:35.181569+0000","last_peered":"2024-05-21T17:41:35.181569+0000","last_clean":"2024-05-21T17:41:35.181569+0000","last_became_active":"2024-05-21T17:41:32.186477+0000","last_became_peered":"2024-05-21T17:41:32.186477+0000","last_unstale":"2024-05-21T17:41:35.181569+0000","last_undegraded":"2024-05-21T17:41:35.181569+0000","last_fullsized":"2024-05-21T17:41:35.181569+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:43:03.647293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503942+0000","last_change":"2024-05-21T17:41:33.214302+0000","last_active":"2024-05-21T17:41:35.503942+0000","last_peered":"2024-05-21T17:41:35.503942+0000","last_clean":"2024-05-21T17:41:35.503942+0000","last_became_active":"2024-05-21T17:41:33.211621+0000","last_became_peered":"2024-05-21T17:41:33.211621+0000","last_unstale":"2024-05-21T17:41:35.503942+0000","last_undegraded":"2024-05-21T17:41:35.503942+0000","last_fullsized":"2024-05-21T17:41:35.503942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:29.332651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181671+0000","last_change":"2024-05-21T17:41:32.185517+0000","last_active":"2024-05-21T17:41:35.181671+0000","last_peered":"2024-05-21T17:41:35.181671+0000","last_clean":"2024-05-21T17:41:35.181671+0000","last_became_active":"2024-05-21T17:41:32.185147+0000","last_became_peered":"2024-05-21T17:41:32.185147+0000","last_unstale":"2024-05-21T17:41:35.181671+0000","last_undegraded":"2024-05-21T17:41:35.181671+0000","last_fullsized":"2024-05-21T17:41:35.181671+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:51:23.199039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195591+0000","last_change":"2024-05-21T17:41:33.192101+0000","last_active":"2024-05-21T17:41:34.195591+0000","last_peered":"2024-05-21T17:41:34.195591+0000","last_clean":"2024-05-21T17:41:34.195591+0000","last_became_active":"2024-05-21T17:41:33.191796+0000","last_became_peered":"2024-05-21T17:41:33.191796+0000","last_unstale":"2024-05-21T17:41:34.195591+0000","last_undegraded":"2024-05-21T17:41:34.195591+0000","last_fullsized":"2024-05-21T17:41:34.195591+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:03:00.661590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505135+0000","last_change":"2024-05-21T17:41:32.197897+0000","last_active":"2024-05-21T17:41:35.505135+0000","last_peered":"2024-05-21T17:41:35.505135+0000","last_clean":"2024-05-21T17:41:35.505135+0000","last_became_active":"2024-05-21T17:41:32.197609+0000","last_became_peered":"2024-05-21T17:41:32.197609+0000","last_unstale":"2024-05-21T17:41:35.505135+0000","last_undegraded":"2024-05-21T17:41:35.505135+0000","last_fullsized":"2024-05-21T17:41:35.505135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:29:06.885892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181127+0000","last_change":"2024-05-21T17:41:33.213002+0000","last_active":"2024-05-21T17:41:35.181127+0000","last_peered":"2024-05-21T17:41:35.181127+0000","last_clean":"2024-05-21T17:41:35.181127+0000","last_became_active":"2024-05-21T17:41:33.212695+0000","last_became_peered":"2024-05-21T17:41:33.212695+0000","last_unstale":"2024-05-21T17:41:35.181127+0000","last_undegraded":"2024-05-21T17:41:35.181127+0000","last_fullsized":"2024-05-21T17:41:35.181127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:48.164464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504834+0000","last_change":"2024-05-21T17:41:32.197048+0000","last_active":"2024-05-21T17:41:35.504834+0000","last_peered":"2024-05-21T17:41:35.504834+0000","last_clean":"2024-05-21T17:41:35.504834+0000","last_became_active":"2024-05-21T17:41:32.196772+0000","last_became_peered":"2024-05-21T17:41:32.196772+0000","last_unstale":"2024-05-21T17:41:35.504834+0000","last_undegraded":"2024-05-21T17:41:35.504834+0000","last_fullsized":"2024-05-21T17:41:35.504834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:40:57.685104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181361+0000","last_change":"2024-05-21T17:41:33.195970+0000","last_active":"2024-05-21T17:41:35.181361+0000","last_peered":"2024-05-21T17:41:35.181361+0000","last_clean":"2024-05-21T17:41:35.181361+0000","last_became_active":"2024-05-21T17:41:33.194971+0000","last_became_peered":"2024-05-21T17:41:33.194971+0000","last_unstale":"2024-05-21T17:41:35.181361+0000","last_undegraded":"2024-05-21T17:41:35.181361+0000","last_fullsized":"2024-05-21T17:41:35.181361+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:06:55.657012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198017+0000","last_change":"2024-05-21T17:41:32.184754+0000","last_active":"2024-05-21T17:41:34.198017+0000","last_peered":"2024-05-21T17:41:34.198017+0000","last_clean":"2024-05-21T17:41:34.198017+0000","last_became_active":"2024-05-21T17:41:32.184457+0000","last_became_peered":"2024-05-21T17:41:32.184457+0000","last_unstale":"2024-05-21T17:41:34.198017+0000","last_undegraded":"2024-05-21T17:41:34.198017+0000","last_fullsized":"2024-05-21T17:41:34.198017+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:33.195952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196321+0000","last_change":"2024-05-21T17:41:33.205458+0000","last_active":"2024-05-21T17:41:34.196321+0000","last_peered":"2024-05-21T17:41:34.196321+0000","last_clean":"2024-05-21T17:41:34.196321+0000","last_became_active":"2024-05-21T17:41:33.204911+0000","last_became_peered":"2024-05-21T17:41:33.204911+0000","last_unstale":"2024-05-21T17:41:34.196321+0000","last_undegraded":"2024-05-21T17:41:34.196321+0000","last_fullsized":"2024-05-21T17:41:34.196321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:37:23.705355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504432+0000","last_change":"2024-05-21T17:41:32.197569+0000","last_active":"2024-05-21T17:41:35.504432+0000","last_peered":"2024-05-21T17:41:35.504432+0000","last_clean":"2024-05-21T17:41:35.504432+0000","last_became_active":"2024-05-21T17:41:32.197242+0000","last_became_peered":"2024-05-21T17:41:32.197242+0000","last_unstale":"2024-05-21T17:41:35.504432+0000","last_undegraded":"2024-05-21T17:41:35.504432+0000","last_fullsized":"2024-05-21T17:41:35.504432+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:54:26.900686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504792+0000","last_change":"2024-05-21T17:41:33.217022+0000","last_active":"2024-05-21T17:41:35.504792+0000","last_peered":"2024-05-21T17:41:35.504792+0000","last_clean":"2024-05-21T17:41:35.504792+0000","last_became_active":"2024-05-21T17:41:33.216705+0000","last_became_peered":"2024-05-21T17:41:33.216705+0000","last_unstale":"2024-05-21T17:41:35.504792+0000","last_undegraded":"2024-05-21T17:41:35.504792+0000","last_fullsized":"2024-05-21T17:41:35.504792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:25:38.189640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504345+0000","last_change":"2024-05-21T17:41:32.194912+0000","last_active":"2024-05-21T17:41:35.504345+0000","last_peered":"2024-05-21T17:41:35.504345+0000","last_clean":"2024-05-21T17:41:35.504345+0000","last_became_active":"2024-05-21T17:41:32.194624+0000","last_became_peered":"2024-05-21T17:41:32.194624+0000","last_unstale":"2024-05-21T17:41:35.504345+0000","last_undegraded":"2024-05-21T17:41:35.504345+0000","last_fullsized":"2024-05-21T17:41:35.504345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:52:59.378552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181533+0000","last_change":"2024-05-21T17:41:33.210044+0000","last_active":"2024-05-21T17:41:35.181533+0000","last_peered":"2024-05-21T17:41:35.181533+0000","last_clean":"2024-05-21T17:41:35.181533+0000","last_became_active":"2024-05-21T17:41:33.209722+0000","last_became_peered":"2024-05-21T17:41:33.209722+0000","last_unstale":"2024-05-21T17:41:35.181533+0000","last_undegraded":"2024-05-21T17:41:35.181533+0000","last_fullsized":"2024-05-21T17:41:35.181533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:37:09.553771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504190+0000","last_change":"2024-05-21T17:41:32.195112+0000","last_active":"2024-05-21T17:41:35.504190+0000","last_peered":"2024-05-21T17:41:35.504190+0000","last_clean":"2024-05-21T17:41:35.504190+0000","last_became_active":"2024-05-21T17:41:32.194765+0000","last_became_peered":"2024-05-21T17:41:32.194765+0000","last_unstale":"2024-05-21T17:41:35.504190+0000","last_undegraded":"2024-05-21T17:41:35.504190+0000","last_fullsized":"2024-05-21T17:41:35.504190+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:11:23.430522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196636+0000","last_change":"2024-05-21T17:41:33.210126+0000","last_active":"2024-05-21T17:41:34.196636+0000","last_peered":"2024-05-21T17:41:34.196636+0000","last_clean":"2024-05-21T17:41:34.196636+0000","last_became_active":"2024-05-21T17:41:33.209679+0000","last_became_peered":"2024-05-21T17:41:33.209679+0000","last_unstale":"2024-05-21T17:41:34.196636+0000","last_undegraded":"2024-05-21T17:41:34.196636+0000","last_fullsized":"2024-05-21T17:41:34.196636+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:14:16.199161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":110,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196448+0000","last_change":"2024-05-21T17:41:27.156765+0000","last_active":"2024-05-21T17:41:34.196448+0000","last_peered":"2024-05-21T17:41:34.196448+0000","last_clean":"2024-05-21T17:41:34.196448+0000","last_became_active":"2024-05-21T17:41:27.156385+0000","last_became_peered":"2024-05-21T17:41:27.156385+0000","last_unstale":"2024-05-21T17:41:34.196448+0000","last_undegraded":"2024-05-21T17:41:34.196448+0000","last_fullsized":"2024-05-21T17:41:34.196448+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504752+0000","last_change":"2024-05-21T17:41:27.156565+0000","last_active":"2024-05-21T17:41:35.504752+0000","last_peered":"2024-05-21T17:41:35.504752+0000","last_clean":"2024-05-21T17:41:35.504752+0000","last_became_active":"2024-05-21T17:41:27.156230+0000","last_became_peered":"2024-05-21T17:41:27.156230+0000","last_unstale":"2024-05-21T17:41:35.504752+0000","last_undegraded":"2024-05-21T17:41:35.504752+0000","last_fullsized":"2024-05-21T17:41:35.504752+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:27.800826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504635+0000","last_change":"2024-05-21T17:41:32.196927+0000","last_active":"2024-05-21T17:41:35.504635+0000","last_peered":"2024-05-21T17:41:35.504635+0000","last_clean":"2024-05-21T17:41:35.504635+0000","last_became_active":"2024-05-21T17:41:32.196462+0000","last_became_peered":"2024-05-21T17:41:32.196462+0000","last_unstale":"2024-05-21T17:41:35.504635+0000","last_undegraded":"2024-05-21T17:41:35.504635+0000","last_fullsized":"2024-05-21T17:41:35.504635+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:47.159182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505094+0000","last_change":"2024-05-21T17:41:33.214046+0000","last_active":"2024-05-21T17:41:35.505094+0000","last_peered":"2024-05-21T17:41:35.505094+0000","last_clean":"2024-05-21T17:41:35.505094+0000","last_became_active":"2024-05-21T17:41:33.213672+0000","last_became_peered":"2024-05-21T17:41:33.213672+0000","last_unstale":"2024-05-21T17:41:35.505094+0000","last_undegraded":"2024-05-21T17:41:35.505094+0000","last_fullsized":"2024-05-21T17:41:35.505094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:12.064652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503362+0000","last_change":"2024-05-21T17:41:27.155004+0000","last_active":"2024-05-21T17:41:35.503362+0000","last_peered":"2024-05-21T17:41:35.503362+0000","last_clean":"2024-05-21T17:41:35.503362+0000","last_became_active":"2024-05-21T17:41:27.154231+0000","last_became_peered":"2024-05-21T17:41:27.154231+0000","last_unstale":"2024-05-21T17:41:35.503362+0000","last_undegraded":"2024-05-21T17:41:35.503362+0000","last_fullsized":"2024-05-21T17:41:35.503362+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:16:06.123434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197552+0000","last_change":"2024-05-21T17:41:32.177330+0000","last_active":"2024-05-21T17:41:34.197552+0000","last_peered":"2024-05-21T17:41:34.197552+0000","last_clean":"2024-05-21T17:41:34.197552+0000","last_became_active":"2024-05-21T17:41:32.176953+0000","last_became_peered":"2024-05-21T17:41:32.176953+0000","last_unstale":"2024-05-21T17:41:34.197552+0000","last_undegraded":"2024-05-21T17:41:34.197552+0000","last_fullsized":"2024-05-21T17:41:34.197552+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:58:39.302037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504924+0000","last_change":"2024-05-21T17:41:33.203272+0000","last_active":"2024-05-21T17:41:35.504924+0000","last_peered":"2024-05-21T17:41:35.504924+0000","last_clean":"2024-05-21T17:41:35.504924+0000","last_became_active":"2024-05-21T17:41:33.202730+0000","last_became_peered":"2024-05-21T17:41:33.202730+0000","last_unstale":"2024-05-21T17:41:35.504924+0000","last_undegraded":"2024-05-21T17:41:35.504924+0000","last_fullsized":"2024-05-21T17:41:35.504924+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:01:35.849261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.180459+0000","last_change":"2024-05-21T17:41:27.159363+0000","last_active":"2024-05-21T17:41:35.180459+0000","last_peered":"2024-05-21T17:41:35.180459+0000","last_clean":"2024-05-21T17:41:35.180459+0000","last_became_active":"2024-05-21T17:41:27.159108+0000","last_became_peered":"2024-05-21T17:41:27.159108+0000","last_unstale":"2024-05-21T17:41:35.180459+0000","last_undegraded":"2024-05-21T17:41:35.180459+0000","last_fullsized":"2024-05-21T17:41:35.180459+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197258+0000","last_change":"2024-05-21T17:41:32.176351+0000","last_active":"2024-05-21T17:41:34.197258+0000","last_peered":"2024-05-21T17:41:34.197258+0000","last_clean":"2024-05-21T17:41:34.197258+0000","last_became_active":"2024-05-21T17:41:32.176008+0000","last_became_peered":"2024-05-21T17:41:32.176008+0000","last_unstale":"2024-05-21T17:41:34.197258+0000","last_undegraded":"2024-05-21T17:41:34.197258+0000","last_fullsized":"2024-05-21T17:41:34.197258+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:07:22.267186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198105+0000","last_change":"2024-05-21T17:41:33.198808+0000","last_active":"2024-05-21T17:41:34.198105+0000","last_peered":"2024-05-21T17:41:34.198105+0000","last_clean":"2024-05-21T17:41:34.198105+0000","last_became_active":"2024-05-21T17:41:33.197653+0000","last_became_peered":"2024-05-21T17:41:33.197653+0000","last_unstale":"2024-05-21T17:41:34.198105+0000","last_undegraded":"2024-05-21T17:41:34.198105+0000","last_fullsized":"2024-05-21T17:41:34.198105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:40:32.331027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503358+0000","last_change":"2024-05-21T17:41:27.154942+0000","last_active":"2024-05-21T17:41:35.503358+0000","last_peered":"2024-05-21T17:41:35.503358+0000","last_clean":"2024-05-21T17:41:35.503358+0000","last_became_active":"2024-05-21T17:41:27.154055+0000","last_became_peered":"2024-05-21T17:41:27.154055+0000","last_unstale":"2024-05-21T17:41:35.503358+0000","last_undegraded":"2024-05-21T17:41:35.503358+0000","last_fullsized":"2024-05-21T17:41:35.503358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:58:33.630777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181371+0000","last_change":"2024-05-21T17:41:32.177942+0000","last_active":"2024-05-21T17:41:35.181371+0000","last_peered":"2024-05-21T17:41:35.181371+0000","last_clean":"2024-05-21T17:41:35.181371+0000","last_became_active":"2024-05-21T17:41:32.177474+0000","last_became_peered":"2024-05-21T17:41:32.177474+0000","last_unstale":"2024-05-21T17:41:35.181371+0000","last_undegraded":"2024-05-21T17:41:35.181371+0000","last_fullsized":"2024-05-21T17:41:35.181371+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:26:32.625845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198276+0000","last_change":"2024-05-21T17:41:33.196975+0000","last_active":"2024-05-21T17:41:34.198276+0000","last_peered":"2024-05-21T17:41:34.198276+0000","last_clean":"2024-05-21T17:41:34.198276+0000","last_became_active":"2024-05-21T17:41:33.196465+0000","last_became_peered":"2024-05-21T17:41:33.196465+0000","last_unstale":"2024-05-21T17:41:34.198276+0000","last_undegraded":"2024-05-21T17:41:34.198276+0000","last_fullsized":"2024-05-21T17:41:34.198276+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:12:57.454391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181373+0000","last_change":"2024-05-21T17:41:32.177494+0000","last_active":"2024-05-21T17:41:35.181373+0000","last_peered":"2024-05-21T17:41:35.181373+0000","last_clean":"2024-05-21T17:41:35.181373+0000","last_became_active":"2024-05-21T17:41:32.177206+0000","last_became_peered":"2024-05-21T17:41:32.177206+0000","last_unstale":"2024-05-21T17:41:35.181373+0000","last_undegraded":"2024-05-21T17:41:35.181373+0000","last_fullsized":"2024-05-21T17:41:35.181373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:58:39.302037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198316+0000","last_change":"2024-05-21T17:41:27.154907+0000","last_active":"2024-05-21T17:41:34.198316+0000","last_peered":"2024-05-21T17:41:34.198316+0000","last_clean":"2024-05-21T17:41:34.198316+0000","last_became_active":"2024-05-21T17:41:27.154659+0000","last_became_peered":"2024-05-21T17:41:27.154659+0000","last_unstale":"2024-05-21T17:41:34.198316+0000","last_undegraded":"2024-05-21T17:41:34.198316+0000","last_fullsized":"2024-05-21T17:41:34.198316+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:28:47.634110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195969+0000","last_change":"2024-05-21T17:41:33.210001+0000","last_active":"2024-05-21T17:41:34.195969+0000","last_peered":"2024-05-21T17:41:34.195969+0000","last_clean":"2024-05-21T17:41:34.195969+0000","last_became_active":"2024-05-21T17:41:33.209694+0000","last_became_peered":"2024-05-21T17:41:33.209694+0000","last_unstale":"2024-05-21T17:41:34.195969+0000","last_undegraded":"2024-05-21T17:41:34.195969+0000","last_fullsized":"2024-05-21T17:41:34.195969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:29:52.943757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198249+0000","last_change":"2024-05-21T17:41:32.177540+0000","last_active":"2024-05-21T17:41:34.198249+0000","last_peered":"2024-05-21T17:41:34.198249+0000","last_clean":"2024-05-21T17:41:34.198249+0000","last_became_active":"2024-05-21T17:41:32.177253+0000","last_became_peered":"2024-05-21T17:41:32.177253+0000","last_unstale":"2024-05-21T17:41:34.198249+0000","last_undegraded":"2024-05-21T17:41:34.198249+0000","last_fullsized":"2024-05-21T17:41:34.198249+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:52.250548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503386+0000","last_change":"2024-05-21T17:41:27.154725+0000","last_active":"2024-05-21T17:41:35.503386+0000","last_peered":"2024-05-21T17:41:35.503386+0000","last_clean":"2024-05-21T17:41:35.503386+0000","last_became_active":"2024-05-21T17:41:27.154339+0000","last_became_peered":"2024-05-21T17:41:27.154339+0000","last_unstale":"2024-05-21T17:41:35.503386+0000","last_undegraded":"2024-05-21T17:41:35.503386+0000","last_fullsized":"2024-05-21T17:41:35.503386+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:47:27.935607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196199+0000","last_change":"2024-05-21T17:41:33.206242+0000","last_active":"2024-05-21T17:41:34.196199+0000","last_peered":"2024-05-21T17:41:34.196199+0000","last_clean":"2024-05-21T17:41:34.196199+0000","last_became_active":"2024-05-21T17:41:33.205945+0000","last_became_peered":"2024-05-21T17:41:33.205945+0000","last_unstale":"2024-05-21T17:41:34.196199+0000","last_undegraded":"2024-05-21T17:41:34.196199+0000","last_fullsized":"2024-05-21T17:41:34.196199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:36:01.886190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505021+0000","last_change":"2024-05-21T17:41:32.193781+0000","last_active":"2024-05-21T17:41:35.505021+0000","last_peered":"2024-05-21T17:41:35.505021+0000","last_clean":"2024-05-21T17:41:35.505021+0000","last_became_active":"2024-05-21T17:41:32.193464+0000","last_became_peered":"2024-05-21T17:41:32.193464+0000","last_unstale":"2024-05-21T17:41:35.505021+0000","last_undegraded":"2024-05-21T17:41:35.505021+0000","last_fullsized":"2024-05-21T17:41:35.505021+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:29:25.307585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198117+0000","last_change":"2024-05-21T17:41:27.156568+0000","last_active":"2024-05-21T17:41:34.198117+0000","last_peered":"2024-05-21T17:41:34.198117+0000","last_clean":"2024-05-21T17:41:34.198117+0000","last_became_active":"2024-05-21T17:41:27.156293+0000","last_became_peered":"2024-05-21T17:41:27.156293+0000","last_unstale":"2024-05-21T17:41:34.198117+0000","last_undegraded":"2024-05-21T17:41:34.198117+0000","last_fullsized":"2024-05-21T17:41:34.198117+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503989+0000","last_change":"2024-05-21T17:41:33.215019+0000","last_active":"2024-05-21T17:41:35.503989+0000","last_peered":"2024-05-21T17:41:35.503989+0000","last_clean":"2024-05-21T17:41:35.503989+0000","last_became_active":"2024-05-21T17:41:33.214714+0000","last_became_peered":"2024-05-21T17:41:33.214714+0000","last_unstale":"2024-05-21T17:41:35.503989+0000","last_undegraded":"2024-05-21T17:41:35.503989+0000","last_fullsized":"2024-05-21T17:41:35.503989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:02:10.347399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.179976+0000","last_change":"2024-05-21T17:41:32.185252+0000","last_active":"2024-05-21T17:41:35.179976+0000","last_peered":"2024-05-21T17:41:35.179976+0000","last_clean":"2024-05-21T17:41:35.179976+0000","last_became_active":"2024-05-21T17:41:32.184893+0000","last_became_peered":"2024-05-21T17:41:32.184893+0000","last_unstale":"2024-05-21T17:41:35.179976+0000","last_undegraded":"2024-05-21T17:41:35.179976+0000","last_fullsized":"2024-05-21T17:41:35.179976+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:33.195952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503430+0000","last_change":"2024-05-21T17:41:27.156830+0000","last_active":"2024-05-21T17:41:35.503430+0000","last_peered":"2024-05-21T17:41:35.503430+0000","last_clean":"2024-05-21T17:41:35.503430+0000","last_became_active":"2024-05-21T17:41:27.156384+0000","last_became_peered":"2024-05-21T17:41:27.156384+0000","last_unstale":"2024-05-21T17:41:35.503430+0000","last_undegraded":"2024-05-21T17:41:35.503430+0000","last_fullsized":"2024-05-21T17:41:35.503430+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:21:15.563350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.180996+0000","last_change":"2024-05-21T17:41:33.197551+0000","last_active":"2024-05-21T17:41:35.180996+0000","last_peered":"2024-05-21T17:41:35.180996+0000","last_clean":"2024-05-21T17:41:35.180996+0000","last_became_active":"2024-05-21T17:41:33.196604+0000","last_became_peered":"2024-05-21T17:41:33.196604+0000","last_unstale":"2024-05-21T17:41:35.180996+0000","last_undegraded":"2024-05-21T17:41:35.180996+0000","last_fullsized":"2024-05-21T17:41:35.180996+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:11:24.435803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181240+0000","last_change":"2024-05-21T17:41:33.210151+0000","last_active":"2024-05-21T17:41:35.181240+0000","last_peered":"2024-05-21T17:41:35.181240+0000","last_clean":"2024-05-21T17:41:35.181240+0000","last_became_active":"2024-05-21T17:41:33.209322+0000","last_became_peered":"2024-05-21T17:41:33.209322+0000","last_unstale":"2024-05-21T17:41:35.181240+0000","last_undegraded":"2024-05-21T17:41:35.181240+0000","last_fullsized":"2024-05-21T17:41:35.181240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:29:07.891173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"15'6","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504664+0000","last_change":"2024-05-21T17:41:32.197388+0000","last_active":"2024-05-21T17:41:35.504664+0000","last_peered":"2024-05-21T17:41:35.504664+0000","last_clean":"2024-05-21T17:41:35.504664+0000","last_became_active":"2024-05-21T17:41:32.196445+0000","last_became_peered":"2024-05-21T17:41:32.196445+0000","last_unstale":"2024-05-21T17:41:35.504664+0000","last_undegraded":"2024-05-21T17:41:35.504664+0000","last_fullsized":"2024-05-21T17:41:35.504664+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:08:36.015445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196326+0000","last_change":"2024-05-21T17:41:33.215210+0000","last_active":"2024-05-21T17:41:34.196326+0000","last_peered":"2024-05-21T17:41:34.196326+0000","last_clean":"2024-05-21T17:41:34.196326+0000","last_became_active":"2024-05-21T17:41:33.214780+0000","last_became_peered":"2024-05-21T17:41:33.214780+0000","last_unstale":"2024-05-21T17:41:34.196326+0000","last_undegraded":"2024-05-21T17:41:34.196326+0000","last_fullsized":"2024-05-21T17:41:34.196326+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:32:22.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"15'3","reported_seq":17,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181189+0000","last_change":"2024-05-21T17:41:32.190183+0000","last_active":"2024-05-21T17:41:35.181189+0000","last_peered":"2024-05-21T17:41:35.181189+0000","last_clean":"2024-05-21T17:41:35.181189+0000","last_became_active":"2024-05-21T17:41:32.189858+0000","last_became_peered":"2024-05-21T17:41:32.189858+0000","last_unstale":"2024-05-21T17:41:35.181189+0000","last_undegraded":"2024-05-21T17:41:35.181189+0000","last_fullsized":"2024-05-21T17:41:35.181189+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:06:50.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181637+0000","last_change":"2024-05-21T17:41:32.183393+0000","last_active":"2024-05-21T17:41:35.181637+0000","last_peered":"2024-05-21T17:41:35.181637+0000","last_clean":"2024-05-21T17:41:35.181637+0000","last_became_active":"2024-05-21T17:41:32.183094+0000","last_became_peered":"2024-05-21T17:41:32.183094+0000","last_unstale":"2024-05-21T17:41:35.181637+0000","last_undegraded":"2024-05-21T17:41:35.181637+0000","last_fullsized":"2024-05-21T17:41:35.181637+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:31:47.729777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197469+0000","last_change":"2024-05-21T17:41:33.203119+0000","last_active":"2024-05-21T17:41:34.197469+0000","last_peered":"2024-05-21T17:41:34.197469+0000","last_clean":"2024-05-21T17:41:34.197469+0000","last_became_active":"2024-05-21T17:41:33.202860+0000","last_became_peered":"2024-05-21T17:41:33.202860+0000","last_unstale":"2024-05-21T17:41:34.197469+0000","last_undegraded":"2024-05-21T17:41:34.197469+0000","last_fullsized":"2024-05-21T17:41:34.197469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:07:06.410067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196732+0000","last_change":"2024-05-21T17:41:32.192104+0000","last_active":"2024-05-21T17:41:34.196732+0000","last_peered":"2024-05-21T17:41:34.196732+0000","last_clean":"2024-05-21T17:41:34.196732+0000","last_became_active":"2024-05-21T17:41:32.191411+0000","last_became_peered":"2024-05-21T17:41:32.191411+0000","last_unstale":"2024-05-21T17:41:34.196732+0000","last_undegraded":"2024-05-21T17:41:34.196732+0000","last_fullsized":"2024-05-21T17:41:34.196732+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:43:03.647293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503768+0000","last_change":"2024-05-21T17:41:33.212185+0000","last_active":"2024-05-21T17:41:35.503768+0000","last_peered":"2024-05-21T17:41:35.503768+0000","last_clean":"2024-05-21T17:41:35.503768+0000","last_became_active":"2024-05-21T17:41:33.211214+0000","last_became_peered":"2024-05-21T17:41:33.211214+0000","last_unstale":"2024-05-21T17:41:35.503768+0000","last_undegraded":"2024-05-21T17:41:35.503768+0000","last_fullsized":"2024-05-21T17:41:35.503768+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:03:55.087212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504945+0000","last_change":"2024-05-21T17:41:33.206294+0000","last_active":"2024-05-21T17:41:35.504945+0000","last_peered":"2024-05-21T17:41:35.504945+0000","last_clean":"2024-05-21T17:41:35.504945+0000","last_became_active":"2024-05-21T17:41:33.205960+0000","last_became_peered":"2024-05-21T17:41:33.205960+0000","last_unstale":"2024-05-21T17:41:35.504945+0000","last_undegraded":"2024-05-21T17:41:35.504945+0000","last_fullsized":"2024-05-21T17:41:35.504945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:06.686603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504088+0000","last_change":"2024-05-21T17:41:32.198839+0000","last_active":"2024-05-21T17:41:35.504088+0000","last_peered":"2024-05-21T17:41:35.504088+0000","last_clean":"2024-05-21T17:41:35.504088+0000","last_became_active":"2024-05-21T17:41:32.198523+0000","last_became_peered":"2024-05-21T17:41:32.198523+0000","last_unstale":"2024-05-21T17:41:35.504088+0000","last_undegraded":"2024-05-21T17:41:35.504088+0000","last_fullsized":"2024-05-21T17:41:35.504088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:07:05.404785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197963+0000","last_change":"2024-05-21T17:41:33.186217+0000","last_active":"2024-05-21T17:41:34.197963+0000","last_peered":"2024-05-21T17:41:34.197963+0000","last_clean":"2024-05-21T17:41:34.197963+0000","last_became_active":"2024-05-21T17:41:33.185935+0000","last_became_peered":"2024-05-21T17:41:33.185935+0000","last_unstale":"2024-05-21T17:41:34.197963+0000","last_undegraded":"2024-05-21T17:41:34.197963+0000","last_fullsized":"2024-05-21T17:41:34.197963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:00:42.905885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195682+0000","last_change":"2024-05-21T17:41:32.191785+0000","last_active":"2024-05-21T17:41:34.195682+0000","last_peered":"2024-05-21T17:41:34.195682+0000","last_clean":"2024-05-21T17:41:34.195682+0000","last_became_active":"2024-05-21T17:41:32.190999+0000","last_became_peered":"2024-05-21T17:41:32.190999+0000","last_unstale":"2024-05-21T17:41:34.195682+0000","last_undegraded":"2024-05-21T17:41:34.195682+0000","last_fullsized":"2024-05-21T17:41:34.195682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:52.250548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504863+0000","last_change":"2024-05-21T17:41:33.213982+0000","last_active":"2024-05-21T17:41:35.504863+0000","last_peered":"2024-05-21T17:41:35.504863+0000","last_clean":"2024-05-21T17:41:35.504863+0000","last_became_active":"2024-05-21T17:41:33.213225+0000","last_became_peered":"2024-05-21T17:41:33.213225+0000","last_unstale":"2024-05-21T17:41:35.504863+0000","last_undegraded":"2024-05-21T17:41:35.504863+0000","last_fullsized":"2024-05-21T17:41:35.504863+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:07:15.129363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196466+0000","last_change":"2024-05-21T17:41:32.182693+0000","last_active":"2024-05-21T17:41:34.196466+0000","last_peered":"2024-05-21T17:41:34.196466+0000","last_clean":"2024-05-21T17:41:34.196466+0000","last_became_active":"2024-05-21T17:41:32.182420+0000","last_became_peered":"2024-05-21T17:41:32.182420+0000","last_unstale":"2024-05-21T17:41:34.196466+0000","last_undegraded":"2024-05-21T17:41:34.196466+0000","last_fullsized":"2024-05-21T17:41:34.196466+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:26:32.625845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198167+0000","last_change":"2024-05-21T17:41:33.198020+0000","last_active":"2024-05-21T17:41:34.198167+0000","last_peered":"2024-05-21T17:41:34.198167+0000","last_clean":"2024-05-21T17:41:34.198167+0000","last_became_active":"2024-05-21T17:41:33.197187+0000","last_became_peered":"2024-05-21T17:41:33.197187+0000","last_unstale":"2024-05-21T17:41:34.198167+0000","last_undegraded":"2024-05-21T17:41:34.198167+0000","last_fullsized":"2024-05-21T17:41:34.198167+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:37:09.553771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504321+0000","last_change":"2024-05-21T17:41:32.197952+0000","last_active":"2024-05-21T17:41:35.504321+0000","last_peered":"2024-05-21T17:41:35.504321+0000","last_clean":"2024-05-21T17:41:35.504321+0000","last_became_active":"2024-05-21T17:41:32.197316+0000","last_became_peered":"2024-05-21T17:41:32.197316+0000","last_unstale":"2024-05-21T17:41:35.504321+0000","last_undegraded":"2024-05-21T17:41:35.504321+0000","last_fullsized":"2024-05-21T17:41:35.504321+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:04:40.497094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181105+0000","last_change":"2024-05-21T17:41:33.196406+0000","last_active":"2024-05-21T17:41:35.181105+0000","last_peered":"2024-05-21T17:41:35.181105+0000","last_clean":"2024-05-21T17:41:35.181105+0000","last_became_active":"2024-05-21T17:41:33.195986+0000","last_became_peered":"2024-05-21T17:41:33.195986+0000","last_unstale":"2024-05-21T17:41:35.181105+0000","last_undegraded":"2024-05-21T17:41:35.181105+0000","last_fullsized":"2024-05-21T17:41:35.181105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:54:27.905968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196455+0000","last_change":"2024-05-21T17:41:32.192119+0000","last_active":"2024-05-21T17:41:34.196455+0000","last_peered":"2024-05-21T17:41:34.196455+0000","last_clean":"2024-05-21T17:41:34.196455+0000","last_became_active":"2024-05-21T17:41:32.191275+0000","last_became_peered":"2024-05-21T17:41:32.191275+0000","last_unstale":"2024-05-21T17:41:34.196455+0000","last_undegraded":"2024-05-21T17:41:34.196455+0000","last_fullsized":"2024-05-21T17:41:34.196455+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:58:39.302037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181217+0000","last_change":"2024-05-21T17:41:33.196519+0000","last_active":"2024-05-21T17:41:35.181217+0000","last_peered":"2024-05-21T17:41:35.181217+0000","last_clean":"2024-05-21T17:41:35.181217+0000","last_became_active":"2024-05-21T17:41:33.194635+0000","last_became_peered":"2024-05-21T17:41:33.194635+0000","last_unstale":"2024-05-21T17:41:35.181217+0000","last_undegraded":"2024-05-21T17:41:35.181217+0000","last_fullsized":"2024-05-21T17:41:35.181217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:00:42.905885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505022+0000","last_change":"2024-05-21T17:41:32.198015+0000","last_active":"2024-05-21T17:41:35.505022+0000","last_peered":"2024-05-21T17:41:35.505022+0000","last_clean":"2024-05-21T17:41:35.505022+0000","last_became_active":"2024-05-21T17:41:32.197483+0000","last_became_peered":"2024-05-21T17:41:32.197483+0000","last_unstale":"2024-05-21T17:41:35.505022+0000","last_undegraded":"2024-05-21T17:41:35.505022+0000","last_fullsized":"2024-05-21T17:41:35.505022+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:00:41.900604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504078+0000","last_change":"2024-05-21T17:41:33.207480+0000","last_active":"2024-05-21T17:41:35.504078+0000","last_peered":"2024-05-21T17:41:35.504078+0000","last_clean":"2024-05-21T17:41:35.504078+0000","last_became_active":"2024-05-21T17:41:33.207211+0000","last_became_peered":"2024-05-21T17:41:33.207211+0000","last_unstale":"2024-05-21T17:41:35.504078+0000","last_undegraded":"2024-05-21T17:41:35.504078+0000","last_fullsized":"2024-05-21T17:41:35.504078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:55:50.762859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198168+0000","last_change":"2024-05-21T17:41:32.180196+0000","last_active":"2024-05-21T17:41:34.198168+0000","last_peered":"2024-05-21T17:41:34.198168+0000","last_clean":"2024-05-21T17:41:34.198168+0000","last_became_active":"2024-05-21T17:41:32.179887+0000","last_became_peered":"2024-05-21T17:41:32.179887+0000","last_unstale":"2024-05-21T17:41:34.198168+0000","last_undegraded":"2024-05-21T17:41:34.198168+0000","last_fullsized":"2024-05-21T17:41:34.198168+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:06:50.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.180828+0000","last_change":"2024-05-21T17:41:33.196424+0000","last_active":"2024-05-21T17:41:35.180828+0000","last_peered":"2024-05-21T17:41:35.180828+0000","last_clean":"2024-05-21T17:41:35.180828+0000","last_became_active":"2024-05-21T17:41:33.196119+0000","last_became_peered":"2024-05-21T17:41:33.196119+0000","last_unstale":"2024-05-21T17:41:35.180828+0000","last_undegraded":"2024-05-21T17:41:35.180828+0000","last_fullsized":"2024-05-21T17:41:35.180828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:29:26.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"15'4","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196681+0000","last_change":"2024-05-21T17:41:32.182802+0000","last_active":"2024-05-21T17:41:34.196681+0000","last_peered":"2024-05-21T17:41:34.196681+0000","last_clean":"2024-05-21T17:41:34.196681+0000","last_became_active":"2024-05-21T17:41:32.182446+0000","last_became_peered":"2024-05-21T17:41:32.182446+0000","last_unstale":"2024-05-21T17:41:34.196681+0000","last_undegraded":"2024-05-21T17:41:34.196681+0000","last_fullsized":"2024-05-21T17:41:34.196681+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:06:50.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17264,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":52460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":642780,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":641696,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":51236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T17:41:41.097 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-21T17:41:41.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.235+0000 7fe8e0bf7640 1 Processor -- start 2024-05-21T17:41:41.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.235+0000 7fe8e0bf7640 1 -- start start 2024-05-21T17:41:41.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.236+0000 7fe8e0bf7640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.236+0000 7fe8e0bf7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe8dc0a70d0 0x7fe8dc150a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.236+0000 7fe8e0bf7640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8dc0a76a0 con 0x7fe8dc14a3e0 2024-05-21T17:41:41.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.236+0000 7fe8e0bf7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fe8dc06fab0 con 0x7fe8dc0a70d0 2024-05-21T17:41:41.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.236+0000 7fe8e0bf7640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fe8dc1496f0 con 0x7fe8dc0a5850 2024-05-21T17:41:41.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.236+0000 7fe8da59c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fe8dc14a3e0 0x7fe8dc0a5eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60736/0 (socket says 172.21.15.3:60736) 2024-05-21T17:41:41.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.236+0000 7fe8da59c640 1 -- 172.21.15.3:0/556280896 learned_addr learned my addr 172.21.15.3:0/556280896 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:41.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.236+0000 7fe8d9d9b640 1 --2- 172.21.15.3:0/556280896 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.237+0000 7fe8d959a640 1 -- 172.21.15.3:0/556280896 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728642109 0 0) 0x7fe8dc0a76a0 con 0x7fe8dc14a3e0 2024-05-21T17:41:41.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.237+0000 7fe8d9d9b640 1 -- 172.21.15.3:0/556280896 >> v2:172.21.15.203:3300/0 conn(0x7fe8dc0a70d0 msgr2=0x7fe8dc150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:41.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.237+0000 7fe8d9d9b640 1 --2- 172.21.15.3:0/556280896 >> v2:172.21.15.203:3300/0 conn(0x7fe8dc0a70d0 0x7fe8dc150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.237+0000 7fe8d9d9b640 1 -- 172.21.15.3:0/556280896 >> v1:172.21.15.3:6789/0 conn(0x7fe8dc14a3e0 legacy=0x7fe8dc0a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.237+0000 7fe8d9d9b640 1 -- 172.21.15.3:0/556280896 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc0b90e0 con 0x7fe8dc0a5850 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.237+0000 7fe8d9d9b640 1 --2- 172.21.15.3:0/556280896 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0b8ba0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00097a0 tx=0x7fe8d002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dacb72f95abfaf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8d959a640 1 -- 172.21.15.3:0/556280896 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00315a0 con 0x7fe8dc0a5850 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8d959a640 1 -- 172.21.15.3:0/556280896 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d0037070 con 0x7fe8dc0a5850 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8d959a640 1 -- 172.21.15.3:0/556280896 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0031960 con 0x7fe8dc0a5850 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/556280896 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 msgr2=0x7fe8dc0b8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8e0bf7640 1 --2- 172.21.15.3:0/556280896 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0b8ba0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00097a0 tx=0x7fe8d002b840 comp rx=0 tx=0).stop 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/556280896 shutdown_connections 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8e0bf7640 1 --2- 172.21.15.3:0/556280896 >> v2:172.21.15.203:3300/0 conn(0x7fe8dc0a70d0 0x7fe8dc150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8e0bf7640 1 --2- 172.21.15.3:0/556280896 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0b8ba0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/556280896 >> 172.21.15.3:0/556280896 conn(0x7fe8dc06d700 msgr2=0x7fe8dc06db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/556280896 shutdown_connections 2024-05-21T17:41:41.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.238+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/556280896 wait complete. 2024-05-21T17:41:41.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.239+0000 7fe8e0bf7640 1 Processor -- start 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.239+0000 7fe8e0bf7640 1 -- start start 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8e0bf7640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8e0bf7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8dc0a70d0 0x7fe8dc0a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8e0bf7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8dc14a3e0 0x7fe8dc1df510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8e0bf7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc071f30 con 0x7fe8dc14a3e0 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8e0bf7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc071db0 con 0x7fe8dc0a70d0 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8e0bf7640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fe8dc150fb0 con 0x7fe8dc0a5850 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8da59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8da59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0a99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41112/0 (socket says 172.21.15.3:41112) 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8dad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8dc14a3e0 0x7fe8dc1df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8da59c640 1 -- 172.21.15.3:0/2403871376 learned_addr learned my addr 172.21.15.3:0/2403871376 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8d9d9b640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8dc0a70d0 0x7fe8dc0a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8da59c640 1 -- 172.21.15.3:0/2403871376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8dc0a70d0 msgr2=0x7fe8dc0a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8da59c640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8dc0a70d0 0x7fe8dc0a79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8da59c640 1 -- 172.21.15.3:0/2403871376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8dc14a3e0 msgr2=0x7fe8dc1df510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8da59c640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8dc14a3e0 0x7fe8dc1df510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8da59c640 1 -- 172.21.15.3:0/2403871376 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc0a84d0 con 0x7fe8dc0a5850 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.240+0000 7fe8dad9d640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8dc14a3e0 0x7fe8dc1df510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.241+0000 7fe8da59c640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0a99a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c410 tx=0x7fe8cc00c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.241+0000 7fe8caffd640 1 -- 172.21.15.3:0/2403871376 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0074e0 con 0x7fe8dc0a5850 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.241+0000 7fe8caffd640 1 -- 172.21.15.3:0/2403871376 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc0047e0 con 0x7fe8dc0a5850 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.241+0000 7fe8caffd640 1 -- 172.21.15.3:0/2403871376 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc004bd0 con 0x7fe8dc0a5850 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.241+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/2403871376 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8dc0a8790 con 0x7fe8dc0a5850 2024-05-21T17:41:41.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.241+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/2403871376 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8dc0a8f30 con 0x7fe8dc0a5850 2024-05-21T17:41:41.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.242+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/2403871376 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8dc0aac50 con 0x7fe8dc0a5850 2024-05-21T17:41:41.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.244+0000 7fe8caffd640 1 -- 172.21.15.3:0/2403871376 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7fe8cc011070 con 0x7fe8dc0a5850 2024-05-21T17:41:41.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.244+0000 7fe8caffd640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:41.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.245+0000 7fe8caffd640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:41.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.245+0000 7fe8caffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:41.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.245+0000 7fe8caffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:41.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.245+0000 7fe8caffd640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fe8bc0af260 0x7fe8bc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.245+0000 7fe8caffd640 1 -- 172.21.15.3:0/2403871376 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7fe8cc0dcb90 con 0x7fe8dc0a5850 2024-05-21T17:41:41.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.245+0000 7fe8d9d9b640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fe8bc0af260 0x7fe8bc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.247 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.246+0000 7fe8d9d9b640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fe8bc0af260 0x7fe8bc0b1720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0036000 tx=0x7fe8d003d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:41.249 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.248+0000 7fe8caffd640 1 -- 172.21.15.3:0/2403871376 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe8cc0a68a0 con 0x7fe8dc0a5850 2024-05-21T17:41:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.437+0000 7fe8e0bf7640 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2024-05-21T17:41:41.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.437+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/2403871376 --> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8dc069a90 con 0x7fe8bc0af260 2024-05-21T17:41:41.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.444+0000 7fe8caffd640 1 -- 172.21.15.3:0/2403871376 <== mgr.4102 v2:172.21.15.203:6824/4157202067 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+268601 (secure 0 0 0) 0x7fe8dc069a90 con 0x7fe8bc0af260 2024-05-21T17:41:41.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.444+0000 7fe8caffd640 20 mgrc handle_command_reply tid 0 r 0 2024-05-21T17:41:41.448 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:41.449 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-21T17:41:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 10 mgrc shutdown 2024-05-21T17:41:41.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/2403871376 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fe8bc0af260 msgr2=0x7fe8bc0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fe8bc0af260 0x7fe8bc0b1720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0036000 tx=0x7fe8d003d040 comp rx=0 tx=0).stop 2024-05-21T17:41:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/2403871376 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 msgr2=0x7fe8dc0a99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0a99a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c410 tx=0x7fe8cc00c8d0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/2403871376 shutdown_connections 2024-05-21T17:41:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fe8bc0af260 0x7fe8bc0b1720 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8dc14a3e0 0x7fe8dc1df510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8dc0a70d0 0x7fe8dc0a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 --2- 172.21.15.3:0/2403871376 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fe8dc0a5850 0x7fe8dc0a99a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/2403871376 >> 172.21.15.3:0/2403871376 conn(0x7fe8dc06d700 msgr2=0x7fe8dc0b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:41.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/2403871376 shutdown_connections 2024-05-21T17:41:41.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.447+0000 7fe8e0bf7640 1 -- 172.21.15.3:0/2403871376 wait complete. 2024-05-21T17:41:41.459 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-21T17:41:39.519719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":197,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":108856,"kb_used_data":768,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377378504,"statfs":{"total":386547056640,"available":386435588096,"internally_reserved":0,"allocated":786432,"data_stored":1388172,"data_compressed":17264,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":6377,"internal_metadata":110618391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.351933"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181633+0000","last_change":"2024-05-21T17:41:33.197173+0000","last_active":"2024-05-21T17:41:35.181633+0000","last_peered":"2024-05-21T17:41:35.181633+0000","last_clean":"2024-05-21T17:41:35.181633+0000","last_became_active":"2024-05-21T17:41:33.196781+0000","last_became_peered":"2024-05-21T17:41:33.196781+0000","last_unstale":"2024-05-21T17:41:35.181633+0000","last_undegraded":"2024-05-21T17:41:35.181633+0000","last_fullsized":"2024-05-21T17:41:35.181633+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:12:57.454391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181522+0000","last_change":"2024-05-21T17:41:33.197965+0000","last_active":"2024-05-21T17:41:35.181522+0000","last_peered":"2024-05-21T17:41:35.181522+0000","last_clean":"2024-05-21T17:41:35.181522+0000","last_became_active":"2024-05-21T17:41:33.197693+0000","last_became_peered":"2024-05-21T17:41:33.197693+0000","last_unstale":"2024-05-21T17:41:35.181522+0000","last_undegraded":"2024-05-21T17:41:35.181522+0000","last_fullsized":"2024-05-21T17:41:35.181522+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:40:32.331027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504514+0000","last_change":"2024-05-21T17:41:33.212529+0000","last_active":"2024-05-21T17:41:35.504514+0000","last_peered":"2024-05-21T17:41:35.504514+0000","last_clean":"2024-05-21T17:41:35.504514+0000","last_became_active":"2024-05-21T17:41:33.211026+0000","last_became_peered":"2024-05-21T17:41:33.211026+0000","last_unstale":"2024-05-21T17:41:35.504514+0000","last_undegraded":"2024-05-21T17:41:35.504514+0000","last_fullsized":"2024-05-21T17:41:35.504514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:15:01.504097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196450+0000","last_change":"2024-05-21T17:41:33.191234+0000","last_active":"2024-05-21T17:41:34.196450+0000","last_peered":"2024-05-21T17:41:34.196450+0000","last_clean":"2024-05-21T17:41:34.196450+0000","last_became_active":"2024-05-21T17:41:33.190138+0000","last_became_peered":"2024-05-21T17:41:33.190138+0000","last_unstale":"2024-05-21T17:41:34.196450+0000","last_undegraded":"2024-05-21T17:41:34.196450+0000","last_fullsized":"2024-05-21T17:41:34.196450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:55:31.958772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504393+0000","last_change":"2024-05-21T17:41:33.203349+0000","last_active":"2024-05-21T17:41:35.504393+0000","last_peered":"2024-05-21T17:41:35.504393+0000","last_clean":"2024-05-21T17:41:35.504393+0000","last_became_active":"2024-05-21T17:41:33.203059+0000","last_became_peered":"2024-05-21T17:41:33.203059+0000","last_unstale":"2024-05-21T17:41:35.504393+0000","last_undegraded":"2024-05-21T17:41:35.504393+0000","last_fullsized":"2024-05-21T17:41:35.504393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:10:26.347216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504144+0000","last_change":"2024-05-21T17:41:33.208230+0000","last_active":"2024-05-21T17:41:35.504144+0000","last_peered":"2024-05-21T17:41:35.504144+0000","last_clean":"2024-05-21T17:41:35.504144+0000","last_became_active":"2024-05-21T17:41:33.207955+0000","last_became_peered":"2024-05-21T17:41:33.207955+0000","last_unstale":"2024-05-21T17:41:35.504144+0000","last_undegraded":"2024-05-21T17:41:35.504144+0000","last_fullsized":"2024-05-21T17:41:35.504144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:42:25.073971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503817+0000","last_change":"2024-05-21T17:41:33.208443+0000","last_active":"2024-05-21T17:41:35.503817+0000","last_peered":"2024-05-21T17:41:35.503817+0000","last_clean":"2024-05-21T17:41:35.503817+0000","last_became_active":"2024-05-21T17:41:33.208038+0000","last_became_peered":"2024-05-21T17:41:33.208038+0000","last_unstale":"2024-05-21T17:41:35.503817+0000","last_undegraded":"2024-05-21T17:41:35.503817+0000","last_fullsized":"2024-05-21T17:41:35.503817+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:14:16.199161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195697+0000","last_change":"2024-05-21T17:41:33.202443+0000","last_active":"2024-05-21T17:41:34.195697+0000","last_peered":"2024-05-21T17:41:34.195697+0000","last_clean":"2024-05-21T17:41:34.195697+0000","last_became_active":"2024-05-21T17:41:33.200436+0000","last_became_peered":"2024-05-21T17:41:33.200436+0000","last_unstale":"2024-05-21T17:41:34.195697+0000","last_undegraded":"2024-05-21T17:41:34.195697+0000","last_fullsized":"2024-05-21T17:41:34.195697+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:41:08.380018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197981+0000","last_change":"2024-05-21T17:41:33.185540+0000","last_active":"2024-05-21T17:41:34.197981+0000","last_peered":"2024-05-21T17:41:34.197981+0000","last_clean":"2024-05-21T17:41:34.197981+0000","last_became_active":"2024-05-21T17:41:33.185269+0000","last_became_peered":"2024-05-21T17:41:33.185269+0000","last_unstale":"2024-05-21T17:41:34.197981+0000","last_undegraded":"2024-05-21T17:41:34.197981+0000","last_fullsized":"2024-05-21T17:41:34.197981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:29:07.891173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196019+0000","last_change":"2024-05-21T17:41:33.204222+0000","last_active":"2024-05-21T17:41:34.196019+0000","last_peered":"2024-05-21T17:41:34.196019+0000","last_clean":"2024-05-21T17:41:34.196019+0000","last_became_active":"2024-05-21T17:41:33.203828+0000","last_became_peered":"2024-05-21T17:41:33.203828+0000","last_unstale":"2024-05-21T17:41:34.196019+0000","last_undegraded":"2024-05-21T17:41:34.196019+0000","last_fullsized":"2024-05-21T17:41:34.196019+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:26:09.846517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196360+0000","last_change":"2024-05-21T17:41:33.190185+0000","last_active":"2024-05-21T17:41:34.196360+0000","last_peered":"2024-05-21T17:41:34.196360+0000","last_clean":"2024-05-21T17:41:34.196360+0000","last_became_active":"2024-05-21T17:41:33.189865+0000","last_became_peered":"2024-05-21T17:41:33.189865+0000","last_unstale":"2024-05-21T17:41:34.196360+0000","last_undegraded":"2024-05-21T17:41:34.196360+0000","last_fullsized":"2024-05-21T17:41:34.196360+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:36:50.231113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181658+0000","last_change":"2024-05-21T17:41:33.212714+0000","last_active":"2024-05-21T17:41:35.181658+0000","last_peered":"2024-05-21T17:41:35.181658+0000","last_clean":"2024-05-21T17:41:35.181658+0000","last_became_active":"2024-05-21T17:41:33.212447+0000","last_became_peered":"2024-05-21T17:41:33.212447+0000","last_unstale":"2024-05-21T17:41:35.181658+0000","last_undegraded":"2024-05-21T17:41:35.181658+0000","last_fullsized":"2024-05-21T17:41:35.181658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:06.343413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197751+0000","last_change":"2024-05-21T17:41:33.194642+0000","last_active":"2024-05-21T17:41:34.197751+0000","last_peered":"2024-05-21T17:41:34.197751+0000","last_clean":"2024-05-21T17:41:34.197751+0000","last_became_active":"2024-05-21T17:41:33.194331+0000","last_became_peered":"2024-05-21T17:41:33.194331+0000","last_unstale":"2024-05-21T17:41:34.197751+0000","last_undegraded":"2024-05-21T17:41:34.197751+0000","last_fullsized":"2024-05-21T17:41:34.197751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:54:27.905968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195573+0000","last_change":"2024-05-21T17:41:33.200546+0000","last_active":"2024-05-21T17:41:34.195573+0000","last_peered":"2024-05-21T17:41:34.195573+0000","last_clean":"2024-05-21T17:41:34.195573+0000","last_became_active":"2024-05-21T17:41:33.199969+0000","last_became_peered":"2024-05-21T17:41:33.199969+0000","last_unstale":"2024-05-21T17:41:34.195573+0000","last_undegraded":"2024-05-21T17:41:34.195573+0000","last_fullsized":"2024-05-21T17:41:34.195573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:06.343413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195553+0000","last_change":"2024-05-21T17:41:33.200283+0000","last_active":"2024-05-21T17:41:34.195553+0000","last_peered":"2024-05-21T17:41:34.195553+0000","last_clean":"2024-05-21T17:41:34.195553+0000","last_became_active":"2024-05-21T17:41:33.199931+0000","last_became_peered":"2024-05-21T17:41:33.199931+0000","last_unstale":"2024-05-21T17:41:34.195553+0000","last_undegraded":"2024-05-21T17:41:34.195553+0000","last_fullsized":"2024-05-21T17:41:34.195553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:12:57.454391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503627+0000","last_change":"2024-05-21T17:41:33.208399+0000","last_active":"2024-05-21T17:41:35.503627+0000","last_peered":"2024-05-21T17:41:35.503627+0000","last_clean":"2024-05-21T17:41:35.503627+0000","last_became_active":"2024-05-21T17:41:33.208104+0000","last_became_peered":"2024-05-21T17:41:33.208104+0000","last_unstale":"2024-05-21T17:41:35.503627+0000","last_undegraded":"2024-05-21T17:41:35.503627+0000","last_fullsized":"2024-05-21T17:41:35.503627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:14:30.272478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196463+0000","last_change":"2024-05-21T17:41:33.200479+0000","last_active":"2024-05-21T17:41:34.196463+0000","last_peered":"2024-05-21T17:41:34.196463+0000","last_clean":"2024-05-21T17:41:34.196463+0000","last_became_active":"2024-05-21T17:41:33.199926+0000","last_became_peered":"2024-05-21T17:41:33.199926+0000","last_unstale":"2024-05-21T17:41:34.196463+0000","last_undegraded":"2024-05-21T17:41:34.196463+0000","last_fullsized":"2024-05-21T17:41:34.196463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:03:55.087212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504684+0000","last_change":"2024-05-21T17:41:33.208923+0000","last_active":"2024-05-21T17:41:35.504684+0000","last_peered":"2024-05-21T17:41:35.504684+0000","last_clean":"2024-05-21T17:41:35.504684+0000","last_became_active":"2024-05-21T17:41:33.208377+0000","last_became_peered":"2024-05-21T17:41:33.208377+0000","last_unstale":"2024-05-21T17:41:35.504684+0000","last_undegraded":"2024-05-21T17:41:35.504684+0000","last_fullsized":"2024-05-21T17:41:35.504684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:35:49.839600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197601+0000","last_change":"2024-05-21T17:41:33.214811+0000","last_active":"2024-05-21T17:41:34.197601+0000","last_peered":"2024-05-21T17:41:34.197601+0000","last_clean":"2024-05-21T17:41:34.197601+0000","last_became_active":"2024-05-21T17:41:33.214502+0000","last_became_peered":"2024-05-21T17:41:33.214502+0000","last_unstale":"2024-05-21T17:41:34.197601+0000","last_undegraded":"2024-05-21T17:41:34.197601+0000","last_fullsized":"2024-05-21T17:41:34.197601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:11:24.435803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196070+0000","last_change":"2024-05-21T17:41:33.190432+0000","last_active":"2024-05-21T17:41:34.196070+0000","last_peered":"2024-05-21T17:41:34.196070+0000","last_clean":"2024-05-21T17:41:34.196070+0000","last_became_active":"2024-05-21T17:41:33.189727+0000","last_became_peered":"2024-05-21T17:41:33.189727+0000","last_unstale":"2024-05-21T17:41:34.196070+0000","last_undegraded":"2024-05-21T17:41:34.196070+0000","last_fullsized":"2024-05-21T17:41:34.196070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:45:43.258694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195941+0000","last_change":"2024-05-21T17:41:33.206591+0000","last_active":"2024-05-21T17:41:34.195941+0000","last_peered":"2024-05-21T17:41:34.195941+0000","last_clean":"2024-05-21T17:41:34.195941+0000","last_became_active":"2024-05-21T17:41:33.205156+0000","last_became_peered":"2024-05-21T17:41:33.205156+0000","last_unstale":"2024-05-21T17:41:34.195941+0000","last_undegraded":"2024-05-21T17:41:34.195941+0000","last_fullsized":"2024-05-21T17:41:34.195941+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:55:26.688263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195772+0000","last_change":"2024-05-21T17:41:33.200833+0000","last_active":"2024-05-21T17:41:34.195772+0000","last_peered":"2024-05-21T17:41:34.195772+0000","last_clean":"2024-05-21T17:41:34.195772+0000","last_became_active":"2024-05-21T17:41:33.200208+0000","last_became_peered":"2024-05-21T17:41:33.200208+0000","last_unstale":"2024-05-21T17:41:34.195772+0000","last_undegraded":"2024-05-21T17:41:34.195772+0000","last_fullsized":"2024-05-21T17:41:34.195772+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:28:20.455810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503916+0000","last_change":"2024-05-21T17:41:33.214524+0000","last_active":"2024-05-21T17:41:35.503916+0000","last_peered":"2024-05-21T17:41:35.503916+0000","last_clean":"2024-05-21T17:41:35.503916+0000","last_became_active":"2024-05-21T17:41:33.211351+0000","last_became_peered":"2024-05-21T17:41:33.211351+0000","last_unstale":"2024-05-21T17:41:35.503916+0000","last_undegraded":"2024-05-21T17:41:35.503916+0000","last_fullsized":"2024-05-21T17:41:35.503916+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:25:58.491664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195655+0000","last_change":"2024-05-21T17:41:33.194325+0000","last_active":"2024-05-21T17:41:34.195655+0000","last_peered":"2024-05-21T17:41:34.195655+0000","last_clean":"2024-05-21T17:41:34.195655+0000","last_became_active":"2024-05-21T17:41:33.193802+0000","last_became_peered":"2024-05-21T17:41:33.193802+0000","last_unstale":"2024-05-21T17:41:34.195655+0000","last_undegraded":"2024-05-21T17:41:34.195655+0000","last_fullsized":"2024-05-21T17:41:34.195655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:38:18.545593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196458+0000","last_change":"2024-05-21T17:41:33.209308+0000","last_active":"2024-05-21T17:41:34.196458+0000","last_peered":"2024-05-21T17:41:34.196458+0000","last_clean":"2024-05-21T17:41:34.196458+0000","last_became_active":"2024-05-21T17:41:33.209014+0000","last_became_peered":"2024-05-21T17:41:33.209014+0000","last_unstale":"2024-05-21T17:41:34.196458+0000","last_undegraded":"2024-05-21T17:41:34.196458+0000","last_fullsized":"2024-05-21T17:41:34.196458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:14:30.272478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197581+0000","last_change":"2024-05-21T17:41:33.197441+0000","last_active":"2024-05-21T17:41:34.197581+0000","last_peered":"2024-05-21T17:41:34.197581+0000","last_clean":"2024-05-21T17:41:34.197581+0000","last_became_active":"2024-05-21T17:41:33.196890+0000","last_became_peered":"2024-05-21T17:41:33.196890+0000","last_unstale":"2024-05-21T17:41:34.197581+0000","last_undegraded":"2024-05-21T17:41:34.197581+0000","last_fullsized":"2024-05-21T17:41:34.197581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:29:26.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505181+0000","last_change":"2024-05-21T17:41:33.214392+0000","last_active":"2024-05-21T17:41:35.505181+0000","last_peered":"2024-05-21T17:41:35.505181+0000","last_clean":"2024-05-21T17:41:35.505181+0000","last_became_active":"2024-05-21T17:41:33.213788+0000","last_became_peered":"2024-05-21T17:41:33.213788+0000","last_unstale":"2024-05-21T17:41:35.505181+0000","last_undegraded":"2024-05-21T17:41:35.505181+0000","last_fullsized":"2024-05-21T17:41:35.505181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:33:54.977766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198327+0000","last_change":"2024-05-21T17:41:33.197602+0000","last_active":"2024-05-21T17:41:34.198327+0000","last_peered":"2024-05-21T17:41:34.198327+0000","last_clean":"2024-05-21T17:41:34.198327+0000","last_became_active":"2024-05-21T17:41:33.196498+0000","last_became_peered":"2024-05-21T17:41:33.196498+0000","last_unstale":"2024-05-21T17:41:34.198327+0000","last_undegraded":"2024-05-21T17:41:34.198327+0000","last_fullsized":"2024-05-21T17:41:34.198327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:29:06.343413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197904+0000","last_change":"2024-05-21T17:41:33.194996+0000","last_active":"2024-05-21T17:41:34.197904+0000","last_peered":"2024-05-21T17:41:34.197904+0000","last_clean":"2024-05-21T17:41:34.197904+0000","last_became_active":"2024-05-21T17:41:33.194676+0000","last_became_peered":"2024-05-21T17:41:33.194676+0000","last_unstale":"2024-05-21T17:41:34.197904+0000","last_undegraded":"2024-05-21T17:41:34.197904+0000","last_fullsized":"2024-05-21T17:41:34.197904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:48.164464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196316+0000","last_change":"2024-05-21T17:41:33.209301+0000","last_active":"2024-05-21T17:41:34.196316+0000","last_peered":"2024-05-21T17:41:34.196316+0000","last_clean":"2024-05-21T17:41:34.196316+0000","last_became_active":"2024-05-21T17:41:33.209030+0000","last_became_peered":"2024-05-21T17:41:33.209030+0000","last_unstale":"2024-05-21T17:41:34.196316+0000","last_undegraded":"2024-05-21T17:41:34.196316+0000","last_fullsized":"2024-05-21T17:41:34.196316+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:46:10.096945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197268+0000","last_change":"2024-05-21T17:41:33.185555+0000","last_active":"2024-05-21T17:41:34.197268+0000","last_peered":"2024-05-21T17:41:34.197268+0000","last_clean":"2024-05-21T17:41:34.197268+0000","last_became_active":"2024-05-21T17:41:33.185206+0000","last_became_peered":"2024-05-21T17:41:33.185206+0000","last_unstale":"2024-05-21T17:41:34.197268+0000","last_undegraded":"2024-05-21T17:41:34.197268+0000","last_fullsized":"2024-05-21T17:41:34.197268+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:53:00.383833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196388+0000","last_change":"2024-05-21T17:41:33.194078+0000","last_active":"2024-05-21T17:41:34.196388+0000","last_peered":"2024-05-21T17:41:34.196388+0000","last_clean":"2024-05-21T17:41:34.196388+0000","last_became_active":"2024-05-21T17:41:33.190990+0000","last_became_peered":"2024-05-21T17:41:33.190990+0000","last_unstale":"2024-05-21T17:41:34.196388+0000","last_undegraded":"2024-05-21T17:41:34.196388+0000","last_fullsized":"2024-05-21T17:41:34.196388+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:23:29.692427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504416+0000","last_change":"2024-05-21T17:41:33.203400+0000","last_active":"2024-05-21T17:41:35.504416+0000","last_peered":"2024-05-21T17:41:35.504416+0000","last_clean":"2024-05-21T17:41:35.504416+0000","last_became_active":"2024-05-21T17:41:33.203117+0000","last_became_peered":"2024-05-21T17:41:33.203117+0000","last_unstale":"2024-05-21T17:41:35.504416+0000","last_undegraded":"2024-05-21T17:41:35.504416+0000","last_fullsized":"2024-05-21T17:41:35.504416+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T21:30:00.800497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504541+0000","last_change":"2024-05-21T17:41:32.194821+0000","last_active":"2024-05-21T17:41:35.504541+0000","last_peered":"2024-05-21T17:41:35.504541+0000","last_clean":"2024-05-21T17:41:35.504541+0000","last_became_active":"2024-05-21T17:41:32.194501+0000","last_became_peered":"2024-05-21T17:41:32.194501+0000","last_unstale":"2024-05-21T17:41:35.504541+0000","last_undegraded":"2024-05-21T17:41:35.504541+0000","last_fullsized":"2024-05-21T17:41:35.504541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:32:31.739397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505001+0000","last_change":"2024-05-21T17:41:33.206606+0000","last_active":"2024-05-21T17:41:35.505001+0000","last_peered":"2024-05-21T17:41:35.505001+0000","last_clean":"2024-05-21T17:41:35.505001+0000","last_became_active":"2024-05-21T17:41:33.206194+0000","last_became_peered":"2024-05-21T17:41:33.206194+0000","last_unstale":"2024-05-21T17:41:35.505001+0000","last_undegraded":"2024-05-21T17:41:35.505001+0000","last_fullsized":"2024-05-21T17:41:35.505001+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:04:01.510328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181288+0000","last_change":"2024-05-21T17:41:32.183395+0000","last_active":"2024-05-21T17:41:35.181288+0000","last_peered":"2024-05-21T17:41:35.181288+0000","last_clean":"2024-05-21T17:41:35.181288+0000","last_became_active":"2024-05-21T17:41:32.183108+0000","last_became_peered":"2024-05-21T17:41:32.183108+0000","last_unstale":"2024-05-21T17:41:35.181288+0000","last_undegraded":"2024-05-21T17:41:35.181288+0000","last_fullsized":"2024-05-21T17:41:35.181288+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:33.195952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504687+0000","last_change":"2024-05-21T17:41:33.198913+0000","last_active":"2024-05-21T17:41:35.504687+0000","last_peered":"2024-05-21T17:41:35.504687+0000","last_clean":"2024-05-21T17:41:35.504687+0000","last_became_active":"2024-05-21T17:41:33.198607+0000","last_became_peered":"2024-05-21T17:41:33.198607+0000","last_unstale":"2024-05-21T17:41:35.504687+0000","last_undegraded":"2024-05-21T17:41:35.504687+0000","last_fullsized":"2024-05-21T17:41:35.504687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:14:01.670496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504299+0000","last_change":"2024-05-21T17:41:32.194461+0000","last_active":"2024-05-21T17:41:35.504299+0000","last_peered":"2024-05-21T17:41:35.504299+0000","last_clean":"2024-05-21T17:41:35.504299+0000","last_became_active":"2024-05-21T17:41:32.194194+0000","last_became_peered":"2024-05-21T17:41:32.194194+0000","last_unstale":"2024-05-21T17:41:35.504299+0000","last_undegraded":"2024-05-21T17:41:35.504299+0000","last_fullsized":"2024-05-21T17:41:35.504299+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:00:15.399735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198049+0000","last_change":"2024-05-21T17:41:33.198965+0000","last_active":"2024-05-21T17:41:34.198049+0000","last_peered":"2024-05-21T17:41:34.198049+0000","last_clean":"2024-05-21T17:41:34.198049+0000","last_became_active":"2024-05-21T17:41:33.197958+0000","last_became_peered":"2024-05-21T17:41:33.197958+0000","last_unstale":"2024-05-21T17:41:34.198049+0000","last_undegraded":"2024-05-21T17:41:34.198049+0000","last_fullsized":"2024-05-21T17:41:34.198049+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:06:55.657012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"15'8","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196258+0000","last_change":"2024-05-21T17:41:32.194159+0000","last_active":"2024-05-21T17:41:34.196258+0000","last_peered":"2024-05-21T17:41:34.196258+0000","last_clean":"2024-05-21T17:41:34.196258+0000","last_became_active":"2024-05-21T17:41:32.193896+0000","last_became_peered":"2024-05-21T17:41:32.193896+0000","last_unstale":"2024-05-21T17:41:34.196258+0000","last_undegraded":"2024-05-21T17:41:34.196258+0000","last_fullsized":"2024-05-21T17:41:34.196258+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:31:47.729777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181569+0000","last_change":"2024-05-21T17:41:32.186754+0000","last_active":"2024-05-21T17:41:35.181569+0000","last_peered":"2024-05-21T17:41:35.181569+0000","last_clean":"2024-05-21T17:41:35.181569+0000","last_became_active":"2024-05-21T17:41:32.186477+0000","last_became_peered":"2024-05-21T17:41:32.186477+0000","last_unstale":"2024-05-21T17:41:35.181569+0000","last_undegraded":"2024-05-21T17:41:35.181569+0000","last_fullsized":"2024-05-21T17:41:35.181569+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:43:03.647293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503942+0000","last_change":"2024-05-21T17:41:33.214302+0000","last_active":"2024-05-21T17:41:35.503942+0000","last_peered":"2024-05-21T17:41:35.503942+0000","last_clean":"2024-05-21T17:41:35.503942+0000","last_became_active":"2024-05-21T17:41:33.211621+0000","last_became_peered":"2024-05-21T17:41:33.211621+0000","last_unstale":"2024-05-21T17:41:35.503942+0000","last_undegraded":"2024-05-21T17:41:35.503942+0000","last_fullsized":"2024-05-21T17:41:35.503942+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:29:29.332651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181671+0000","last_change":"2024-05-21T17:41:32.185517+0000","last_active":"2024-05-21T17:41:35.181671+0000","last_peered":"2024-05-21T17:41:35.181671+0000","last_clean":"2024-05-21T17:41:35.181671+0000","last_became_active":"2024-05-21T17:41:32.185147+0000","last_became_peered":"2024-05-21T17:41:32.185147+0000","last_unstale":"2024-05-21T17:41:35.181671+0000","last_undegraded":"2024-05-21T17:41:35.181671+0000","last_fullsized":"2024-05-21T17:41:35.181671+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:51:23.199039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195591+0000","last_change":"2024-05-21T17:41:33.192101+0000","last_active":"2024-05-21T17:41:34.195591+0000","last_peered":"2024-05-21T17:41:34.195591+0000","last_clean":"2024-05-21T17:41:34.195591+0000","last_became_active":"2024-05-21T17:41:33.191796+0000","last_became_peered":"2024-05-21T17:41:33.191796+0000","last_unstale":"2024-05-21T17:41:34.195591+0000","last_undegraded":"2024-05-21T17:41:34.195591+0000","last_fullsized":"2024-05-21T17:41:34.195591+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:03:00.661590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505135+0000","last_change":"2024-05-21T17:41:32.197897+0000","last_active":"2024-05-21T17:41:35.505135+0000","last_peered":"2024-05-21T17:41:35.505135+0000","last_clean":"2024-05-21T17:41:35.505135+0000","last_became_active":"2024-05-21T17:41:32.197609+0000","last_became_peered":"2024-05-21T17:41:32.197609+0000","last_unstale":"2024-05-21T17:41:35.505135+0000","last_undegraded":"2024-05-21T17:41:35.505135+0000","last_fullsized":"2024-05-21T17:41:35.505135+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:29:06.885892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181127+0000","last_change":"2024-05-21T17:41:33.213002+0000","last_active":"2024-05-21T17:41:35.181127+0000","last_peered":"2024-05-21T17:41:35.181127+0000","last_clean":"2024-05-21T17:41:35.181127+0000","last_became_active":"2024-05-21T17:41:33.212695+0000","last_became_peered":"2024-05-21T17:41:33.212695+0000","last_unstale":"2024-05-21T17:41:35.181127+0000","last_undegraded":"2024-05-21T17:41:35.181127+0000","last_fullsized":"2024-05-21T17:41:35.181127+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:48.164464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504834+0000","last_change":"2024-05-21T17:41:32.197048+0000","last_active":"2024-05-21T17:41:35.504834+0000","last_peered":"2024-05-21T17:41:35.504834+0000","last_clean":"2024-05-21T17:41:35.504834+0000","last_became_active":"2024-05-21T17:41:32.196772+0000","last_became_peered":"2024-05-21T17:41:32.196772+0000","last_unstale":"2024-05-21T17:41:35.504834+0000","last_undegraded":"2024-05-21T17:41:35.504834+0000","last_fullsized":"2024-05-21T17:41:35.504834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:40:57.685104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181361+0000","last_change":"2024-05-21T17:41:33.195970+0000","last_active":"2024-05-21T17:41:35.181361+0000","last_peered":"2024-05-21T17:41:35.181361+0000","last_clean":"2024-05-21T17:41:35.181361+0000","last_became_active":"2024-05-21T17:41:33.194971+0000","last_became_peered":"2024-05-21T17:41:33.194971+0000","last_unstale":"2024-05-21T17:41:35.181361+0000","last_undegraded":"2024-05-21T17:41:35.181361+0000","last_fullsized":"2024-05-21T17:41:35.181361+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:06:55.657012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198017+0000","last_change":"2024-05-21T17:41:32.184754+0000","last_active":"2024-05-21T17:41:34.198017+0000","last_peered":"2024-05-21T17:41:34.198017+0000","last_clean":"2024-05-21T17:41:34.198017+0000","last_became_active":"2024-05-21T17:41:32.184457+0000","last_became_peered":"2024-05-21T17:41:32.184457+0000","last_unstale":"2024-05-21T17:41:34.198017+0000","last_undegraded":"2024-05-21T17:41:34.198017+0000","last_fullsized":"2024-05-21T17:41:34.198017+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:33.195952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196321+0000","last_change":"2024-05-21T17:41:33.205458+0000","last_active":"2024-05-21T17:41:34.196321+0000","last_peered":"2024-05-21T17:41:34.196321+0000","last_clean":"2024-05-21T17:41:34.196321+0000","last_became_active":"2024-05-21T17:41:33.204911+0000","last_became_peered":"2024-05-21T17:41:33.204911+0000","last_unstale":"2024-05-21T17:41:34.196321+0000","last_undegraded":"2024-05-21T17:41:34.196321+0000","last_fullsized":"2024-05-21T17:41:34.196321+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:37:23.705355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504432+0000","last_change":"2024-05-21T17:41:32.197569+0000","last_active":"2024-05-21T17:41:35.504432+0000","last_peered":"2024-05-21T17:41:35.504432+0000","last_clean":"2024-05-21T17:41:35.504432+0000","last_became_active":"2024-05-21T17:41:32.197242+0000","last_became_peered":"2024-05-21T17:41:32.197242+0000","last_unstale":"2024-05-21T17:41:35.504432+0000","last_undegraded":"2024-05-21T17:41:35.504432+0000","last_fullsized":"2024-05-21T17:41:35.504432+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:54:26.900686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504792+0000","last_change":"2024-05-21T17:41:33.217022+0000","last_active":"2024-05-21T17:41:35.504792+0000","last_peered":"2024-05-21T17:41:35.504792+0000","last_clean":"2024-05-21T17:41:35.504792+0000","last_became_active":"2024-05-21T17:41:33.216705+0000","last_became_peered":"2024-05-21T17:41:33.216705+0000","last_unstale":"2024-05-21T17:41:35.504792+0000","last_undegraded":"2024-05-21T17:41:35.504792+0000","last_fullsized":"2024-05-21T17:41:35.504792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:25:38.189640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504345+0000","last_change":"2024-05-21T17:41:32.194912+0000","last_active":"2024-05-21T17:41:35.504345+0000","last_peered":"2024-05-21T17:41:35.504345+0000","last_clean":"2024-05-21T17:41:35.504345+0000","last_became_active":"2024-05-21T17:41:32.194624+0000","last_became_peered":"2024-05-21T17:41:32.194624+0000","last_unstale":"2024-05-21T17:41:35.504345+0000","last_undegraded":"2024-05-21T17:41:35.504345+0000","last_fullsized":"2024-05-21T17:41:35.504345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:52:59.378552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181533+0000","last_change":"2024-05-21T17:41:33.210044+0000","last_active":"2024-05-21T17:41:35.181533+0000","last_peered":"2024-05-21T17:41:35.181533+0000","last_clean":"2024-05-21T17:41:35.181533+0000","last_became_active":"2024-05-21T17:41:33.209722+0000","last_became_peered":"2024-05-21T17:41:33.209722+0000","last_unstale":"2024-05-21T17:41:35.181533+0000","last_undegraded":"2024-05-21T17:41:35.181533+0000","last_fullsized":"2024-05-21T17:41:35.181533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:37:09.553771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504190+0000","last_change":"2024-05-21T17:41:32.195112+0000","last_active":"2024-05-21T17:41:35.504190+0000","last_peered":"2024-05-21T17:41:35.504190+0000","last_clean":"2024-05-21T17:41:35.504190+0000","last_became_active":"2024-05-21T17:41:32.194765+0000","last_became_peered":"2024-05-21T17:41:32.194765+0000","last_unstale":"2024-05-21T17:41:35.504190+0000","last_undegraded":"2024-05-21T17:41:35.504190+0000","last_fullsized":"2024-05-21T17:41:35.504190+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:11:23.430522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196636+0000","last_change":"2024-05-21T17:41:33.210126+0000","last_active":"2024-05-21T17:41:34.196636+0000","last_peered":"2024-05-21T17:41:34.196636+0000","last_clean":"2024-05-21T17:41:34.196636+0000","last_became_active":"2024-05-21T17:41:33.209679+0000","last_became_peered":"2024-05-21T17:41:33.209679+0000","last_unstale":"2024-05-21T17:41:34.196636+0000","last_undegraded":"2024-05-21T17:41:34.196636+0000","last_fullsized":"2024-05-21T17:41:34.196636+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:14:16.199161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":110,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196448+0000","last_change":"2024-05-21T17:41:27.156765+0000","last_active":"2024-05-21T17:41:34.196448+0000","last_peered":"2024-05-21T17:41:34.196448+0000","last_clean":"2024-05-21T17:41:34.196448+0000","last_became_active":"2024-05-21T17:41:27.156385+0000","last_became_peered":"2024-05-21T17:41:27.156385+0000","last_unstale":"2024-05-21T17:41:34.196448+0000","last_undegraded":"2024-05-21T17:41:34.196448+0000","last_fullsized":"2024-05-21T17:41:34.196448+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504752+0000","last_change":"2024-05-21T17:41:27.156565+0000","last_active":"2024-05-21T17:41:35.504752+0000","last_peered":"2024-05-21T17:41:35.504752+0000","last_clean":"2024-05-21T17:41:35.504752+0000","last_became_active":"2024-05-21T17:41:27.156230+0000","last_became_peered":"2024-05-21T17:41:27.156230+0000","last_unstale":"2024-05-21T17:41:35.504752+0000","last_undegraded":"2024-05-21T17:41:35.504752+0000","last_fullsized":"2024-05-21T17:41:35.504752+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:27.800826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504635+0000","last_change":"2024-05-21T17:41:32.196927+0000","last_active":"2024-05-21T17:41:35.504635+0000","last_peered":"2024-05-21T17:41:35.504635+0000","last_clean":"2024-05-21T17:41:35.504635+0000","last_became_active":"2024-05-21T17:41:32.196462+0000","last_became_peered":"2024-05-21T17:41:32.196462+0000","last_unstale":"2024-05-21T17:41:35.504635+0000","last_undegraded":"2024-05-21T17:41:35.504635+0000","last_fullsized":"2024-05-21T17:41:35.504635+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:49:47.159182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505094+0000","last_change":"2024-05-21T17:41:33.214046+0000","last_active":"2024-05-21T17:41:35.505094+0000","last_peered":"2024-05-21T17:41:35.505094+0000","last_clean":"2024-05-21T17:41:35.505094+0000","last_became_active":"2024-05-21T17:41:33.213672+0000","last_became_peered":"2024-05-21T17:41:33.213672+0000","last_unstale":"2024-05-21T17:41:35.505094+0000","last_undegraded":"2024-05-21T17:41:35.505094+0000","last_fullsized":"2024-05-21T17:41:35.505094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:12.064652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503362+0000","last_change":"2024-05-21T17:41:27.155004+0000","last_active":"2024-05-21T17:41:35.503362+0000","last_peered":"2024-05-21T17:41:35.503362+0000","last_clean":"2024-05-21T17:41:35.503362+0000","last_became_active":"2024-05-21T17:41:27.154231+0000","last_became_peered":"2024-05-21T17:41:27.154231+0000","last_unstale":"2024-05-21T17:41:35.503362+0000","last_undegraded":"2024-05-21T17:41:35.503362+0000","last_fullsized":"2024-05-21T17:41:35.503362+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:16:06.123434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197552+0000","last_change":"2024-05-21T17:41:32.177330+0000","last_active":"2024-05-21T17:41:34.197552+0000","last_peered":"2024-05-21T17:41:34.197552+0000","last_clean":"2024-05-21T17:41:34.197552+0000","last_became_active":"2024-05-21T17:41:32.176953+0000","last_became_peered":"2024-05-21T17:41:32.176953+0000","last_unstale":"2024-05-21T17:41:34.197552+0000","last_undegraded":"2024-05-21T17:41:34.197552+0000","last_fullsized":"2024-05-21T17:41:34.197552+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:58:39.302037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504924+0000","last_change":"2024-05-21T17:41:33.203272+0000","last_active":"2024-05-21T17:41:35.504924+0000","last_peered":"2024-05-21T17:41:35.504924+0000","last_clean":"2024-05-21T17:41:35.504924+0000","last_became_active":"2024-05-21T17:41:33.202730+0000","last_became_peered":"2024-05-21T17:41:33.202730+0000","last_unstale":"2024-05-21T17:41:35.504924+0000","last_undegraded":"2024-05-21T17:41:35.504924+0000","last_fullsized":"2024-05-21T17:41:35.504924+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T22:01:35.849261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.180459+0000","last_change":"2024-05-21T17:41:27.159363+0000","last_active":"2024-05-21T17:41:35.180459+0000","last_peered":"2024-05-21T17:41:35.180459+0000","last_clean":"2024-05-21T17:41:35.180459+0000","last_became_active":"2024-05-21T17:41:27.159108+0000","last_became_peered":"2024-05-21T17:41:27.159108+0000","last_unstale":"2024-05-21T17:41:35.180459+0000","last_undegraded":"2024-05-21T17:41:35.180459+0000","last_fullsized":"2024-05-21T17:41:35.180459+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197258+0000","last_change":"2024-05-21T17:41:32.176351+0000","last_active":"2024-05-21T17:41:34.197258+0000","last_peered":"2024-05-21T17:41:34.197258+0000","last_clean":"2024-05-21T17:41:34.197258+0000","last_became_active":"2024-05-21T17:41:32.176008+0000","last_became_peered":"2024-05-21T17:41:32.176008+0000","last_unstale":"2024-05-21T17:41:34.197258+0000","last_undegraded":"2024-05-21T17:41:34.197258+0000","last_fullsized":"2024-05-21T17:41:34.197258+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:07:22.267186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198105+0000","last_change":"2024-05-21T17:41:33.198808+0000","last_active":"2024-05-21T17:41:34.198105+0000","last_peered":"2024-05-21T17:41:34.198105+0000","last_clean":"2024-05-21T17:41:34.198105+0000","last_became_active":"2024-05-21T17:41:33.197653+0000","last_became_peered":"2024-05-21T17:41:33.197653+0000","last_unstale":"2024-05-21T17:41:34.198105+0000","last_undegraded":"2024-05-21T17:41:34.198105+0000","last_fullsized":"2024-05-21T17:41:34.198105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:40:32.331027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503358+0000","last_change":"2024-05-21T17:41:27.154942+0000","last_active":"2024-05-21T17:41:35.503358+0000","last_peered":"2024-05-21T17:41:35.503358+0000","last_clean":"2024-05-21T17:41:35.503358+0000","last_became_active":"2024-05-21T17:41:27.154055+0000","last_became_peered":"2024-05-21T17:41:27.154055+0000","last_unstale":"2024-05-21T17:41:35.503358+0000","last_undegraded":"2024-05-21T17:41:35.503358+0000","last_fullsized":"2024-05-21T17:41:35.503358+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:58:33.630777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181371+0000","last_change":"2024-05-21T17:41:32.177942+0000","last_active":"2024-05-21T17:41:35.181371+0000","last_peered":"2024-05-21T17:41:35.181371+0000","last_clean":"2024-05-21T17:41:35.181371+0000","last_became_active":"2024-05-21T17:41:32.177474+0000","last_became_peered":"2024-05-21T17:41:32.177474+0000","last_unstale":"2024-05-21T17:41:35.181371+0000","last_undegraded":"2024-05-21T17:41:35.181371+0000","last_fullsized":"2024-05-21T17:41:35.181371+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:26:32.625845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198276+0000","last_change":"2024-05-21T17:41:33.196975+0000","last_active":"2024-05-21T17:41:34.198276+0000","last_peered":"2024-05-21T17:41:34.198276+0000","last_clean":"2024-05-21T17:41:34.198276+0000","last_became_active":"2024-05-21T17:41:33.196465+0000","last_became_peered":"2024-05-21T17:41:33.196465+0000","last_unstale":"2024-05-21T17:41:34.198276+0000","last_undegraded":"2024-05-21T17:41:34.198276+0000","last_fullsized":"2024-05-21T17:41:34.198276+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:12:57.454391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181373+0000","last_change":"2024-05-21T17:41:32.177494+0000","last_active":"2024-05-21T17:41:35.181373+0000","last_peered":"2024-05-21T17:41:35.181373+0000","last_clean":"2024-05-21T17:41:35.181373+0000","last_became_active":"2024-05-21T17:41:32.177206+0000","last_became_peered":"2024-05-21T17:41:32.177206+0000","last_unstale":"2024-05-21T17:41:35.181373+0000","last_undegraded":"2024-05-21T17:41:35.181373+0000","last_fullsized":"2024-05-21T17:41:35.181373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:58:39.302037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198316+0000","last_change":"2024-05-21T17:41:27.154907+0000","last_active":"2024-05-21T17:41:34.198316+0000","last_peered":"2024-05-21T17:41:34.198316+0000","last_clean":"2024-05-21T17:41:34.198316+0000","last_became_active":"2024-05-21T17:41:27.154659+0000","last_became_peered":"2024-05-21T17:41:27.154659+0000","last_unstale":"2024-05-21T17:41:34.198316+0000","last_undegraded":"2024-05-21T17:41:34.198316+0000","last_fullsized":"2024-05-21T17:41:34.198316+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:28:47.634110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195969+0000","last_change":"2024-05-21T17:41:33.210001+0000","last_active":"2024-05-21T17:41:34.195969+0000","last_peered":"2024-05-21T17:41:34.195969+0000","last_clean":"2024-05-21T17:41:34.195969+0000","last_became_active":"2024-05-21T17:41:33.209694+0000","last_became_peered":"2024-05-21T17:41:33.209694+0000","last_unstale":"2024-05-21T17:41:34.195969+0000","last_undegraded":"2024-05-21T17:41:34.195969+0000","last_fullsized":"2024-05-21T17:41:34.195969+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T05:29:52.943757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198249+0000","last_change":"2024-05-21T17:41:32.177540+0000","last_active":"2024-05-21T17:41:34.198249+0000","last_peered":"2024-05-21T17:41:34.198249+0000","last_clean":"2024-05-21T17:41:34.198249+0000","last_became_active":"2024-05-21T17:41:32.177253+0000","last_became_peered":"2024-05-21T17:41:32.177253+0000","last_unstale":"2024-05-21T17:41:34.198249+0000","last_undegraded":"2024-05-21T17:41:34.198249+0000","last_fullsized":"2024-05-21T17:41:34.198249+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:52.250548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503386+0000","last_change":"2024-05-21T17:41:27.154725+0000","last_active":"2024-05-21T17:41:35.503386+0000","last_peered":"2024-05-21T17:41:35.503386+0000","last_clean":"2024-05-21T17:41:35.503386+0000","last_became_active":"2024-05-21T17:41:27.154339+0000","last_became_peered":"2024-05-21T17:41:27.154339+0000","last_unstale":"2024-05-21T17:41:35.503386+0000","last_undegraded":"2024-05-21T17:41:35.503386+0000","last_fullsized":"2024-05-21T17:41:35.503386+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:47:27.935607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196199+0000","last_change":"2024-05-21T17:41:33.206242+0000","last_active":"2024-05-21T17:41:34.196199+0000","last_peered":"2024-05-21T17:41:34.196199+0000","last_clean":"2024-05-21T17:41:34.196199+0000","last_became_active":"2024-05-21T17:41:33.205945+0000","last_became_peered":"2024-05-21T17:41:33.205945+0000","last_unstale":"2024-05-21T17:41:34.196199+0000","last_undegraded":"2024-05-21T17:41:34.196199+0000","last_fullsized":"2024-05-21T17:41:34.196199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:36:01.886190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505021+0000","last_change":"2024-05-21T17:41:32.193781+0000","last_active":"2024-05-21T17:41:35.505021+0000","last_peered":"2024-05-21T17:41:35.505021+0000","last_clean":"2024-05-21T17:41:35.505021+0000","last_became_active":"2024-05-21T17:41:32.193464+0000","last_became_peered":"2024-05-21T17:41:32.193464+0000","last_unstale":"2024-05-21T17:41:35.505021+0000","last_undegraded":"2024-05-21T17:41:35.505021+0000","last_fullsized":"2024-05-21T17:41:35.505021+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:29:25.307585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198117+0000","last_change":"2024-05-21T17:41:27.156568+0000","last_active":"2024-05-21T17:41:34.198117+0000","last_peered":"2024-05-21T17:41:34.198117+0000","last_clean":"2024-05-21T17:41:34.198117+0000","last_became_active":"2024-05-21T17:41:27.156293+0000","last_became_peered":"2024-05-21T17:41:27.156293+0000","last_unstale":"2024-05-21T17:41:34.198117+0000","last_undegraded":"2024-05-21T17:41:34.198117+0000","last_fullsized":"2024-05-21T17:41:34.198117+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:01:47.989786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503989+0000","last_change":"2024-05-21T17:41:33.215019+0000","last_active":"2024-05-21T17:41:35.503989+0000","last_peered":"2024-05-21T17:41:35.503989+0000","last_clean":"2024-05-21T17:41:35.503989+0000","last_became_active":"2024-05-21T17:41:33.214714+0000","last_became_peered":"2024-05-21T17:41:33.214714+0000","last_unstale":"2024-05-21T17:41:35.503989+0000","last_undegraded":"2024-05-21T17:41:35.503989+0000","last_fullsized":"2024-05-21T17:41:35.503989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:02:10.347399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.179976+0000","last_change":"2024-05-21T17:41:32.185252+0000","last_active":"2024-05-21T17:41:35.179976+0000","last_peered":"2024-05-21T17:41:35.179976+0000","last_clean":"2024-05-21T17:41:35.179976+0000","last_became_active":"2024-05-21T17:41:32.184893+0000","last_became_peered":"2024-05-21T17:41:32.184893+0000","last_unstale":"2024-05-21T17:41:35.179976+0000","last_undegraded":"2024-05-21T17:41:35.179976+0000","last_fullsized":"2024-05-21T17:41:35.179976+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:53:33.195952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503430+0000","last_change":"2024-05-21T17:41:27.156830+0000","last_active":"2024-05-21T17:41:35.503430+0000","last_peered":"2024-05-21T17:41:35.503430+0000","last_clean":"2024-05-21T17:41:35.503430+0000","last_became_active":"2024-05-21T17:41:27.156384+0000","last_became_peered":"2024-05-21T17:41:27.156384+0000","last_unstale":"2024-05-21T17:41:35.503430+0000","last_undegraded":"2024-05-21T17:41:35.503430+0000","last_fullsized":"2024-05-21T17:41:35.503430+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:26.138208+0000","last_clean_scrub_stamp":"2024-05-21T17:41:26.138208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:21:15.563350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.180996+0000","last_change":"2024-05-21T17:41:33.197551+0000","last_active":"2024-05-21T17:41:35.180996+0000","last_peered":"2024-05-21T17:41:35.180996+0000","last_clean":"2024-05-21T17:41:35.180996+0000","last_became_active":"2024-05-21T17:41:33.196604+0000","last_became_peered":"2024-05-21T17:41:33.196604+0000","last_unstale":"2024-05-21T17:41:35.180996+0000","last_undegraded":"2024-05-21T17:41:35.180996+0000","last_fullsized":"2024-05-21T17:41:35.180996+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:11:24.435803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181240+0000","last_change":"2024-05-21T17:41:33.210151+0000","last_active":"2024-05-21T17:41:35.181240+0000","last_peered":"2024-05-21T17:41:35.181240+0000","last_clean":"2024-05-21T17:41:35.181240+0000","last_became_active":"2024-05-21T17:41:33.209322+0000","last_became_peered":"2024-05-21T17:41:33.209322+0000","last_unstale":"2024-05-21T17:41:35.181240+0000","last_undegraded":"2024-05-21T17:41:35.181240+0000","last_fullsized":"2024-05-21T17:41:35.181240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T00:29:07.891173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"15'6","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504664+0000","last_change":"2024-05-21T17:41:32.197388+0000","last_active":"2024-05-21T17:41:35.504664+0000","last_peered":"2024-05-21T17:41:35.504664+0000","last_clean":"2024-05-21T17:41:35.504664+0000","last_became_active":"2024-05-21T17:41:32.196445+0000","last_became_peered":"2024-05-21T17:41:32.196445+0000","last_unstale":"2024-05-21T17:41:35.504664+0000","last_undegraded":"2024-05-21T17:41:35.504664+0000","last_fullsized":"2024-05-21T17:41:35.504664+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:08:36.015445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196326+0000","last_change":"2024-05-21T17:41:33.215210+0000","last_active":"2024-05-21T17:41:34.196326+0000","last_peered":"2024-05-21T17:41:34.196326+0000","last_clean":"2024-05-21T17:41:34.196326+0000","last_became_active":"2024-05-21T17:41:33.214780+0000","last_became_peered":"2024-05-21T17:41:33.214780+0000","last_unstale":"2024-05-21T17:41:34.196326+0000","last_undegraded":"2024-05-21T17:41:34.196326+0000","last_fullsized":"2024-05-21T17:41:34.196326+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:32:22.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"15'3","reported_seq":17,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181189+0000","last_change":"2024-05-21T17:41:32.190183+0000","last_active":"2024-05-21T17:41:35.181189+0000","last_peered":"2024-05-21T17:41:35.181189+0000","last_clean":"2024-05-21T17:41:35.181189+0000","last_became_active":"2024-05-21T17:41:32.189858+0000","last_became_peered":"2024-05-21T17:41:32.189858+0000","last_unstale":"2024-05-21T17:41:35.181189+0000","last_undegraded":"2024-05-21T17:41:35.181189+0000","last_fullsized":"2024-05-21T17:41:35.181189+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:06:50.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181637+0000","last_change":"2024-05-21T17:41:32.183393+0000","last_active":"2024-05-21T17:41:35.181637+0000","last_peered":"2024-05-21T17:41:35.181637+0000","last_clean":"2024-05-21T17:41:35.181637+0000","last_became_active":"2024-05-21T17:41:32.183094+0000","last_became_peered":"2024-05-21T17:41:32.183094+0000","last_unstale":"2024-05-21T17:41:35.181637+0000","last_undegraded":"2024-05-21T17:41:35.181637+0000","last_fullsized":"2024-05-21T17:41:35.181637+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:31:47.729777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197469+0000","last_change":"2024-05-21T17:41:33.203119+0000","last_active":"2024-05-21T17:41:34.197469+0000","last_peered":"2024-05-21T17:41:34.197469+0000","last_clean":"2024-05-21T17:41:34.197469+0000","last_became_active":"2024-05-21T17:41:33.202860+0000","last_became_peered":"2024-05-21T17:41:33.202860+0000","last_unstale":"2024-05-21T17:41:34.197469+0000","last_undegraded":"2024-05-21T17:41:34.197469+0000","last_fullsized":"2024-05-21T17:41:34.197469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:07:06.410067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196732+0000","last_change":"2024-05-21T17:41:32.192104+0000","last_active":"2024-05-21T17:41:34.196732+0000","last_peered":"2024-05-21T17:41:34.196732+0000","last_clean":"2024-05-21T17:41:34.196732+0000","last_became_active":"2024-05-21T17:41:32.191411+0000","last_became_peered":"2024-05-21T17:41:32.191411+0000","last_unstale":"2024-05-21T17:41:34.196732+0000","last_undegraded":"2024-05-21T17:41:34.196732+0000","last_fullsized":"2024-05-21T17:41:34.196732+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T02:43:03.647293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.503768+0000","last_change":"2024-05-21T17:41:33.212185+0000","last_active":"2024-05-21T17:41:35.503768+0000","last_peered":"2024-05-21T17:41:35.503768+0000","last_clean":"2024-05-21T17:41:35.503768+0000","last_became_active":"2024-05-21T17:41:33.211214+0000","last_became_peered":"2024-05-21T17:41:33.211214+0000","last_unstale":"2024-05-21T17:41:35.503768+0000","last_undegraded":"2024-05-21T17:41:35.503768+0000","last_fullsized":"2024-05-21T17:41:35.503768+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:03:55.087212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504945+0000","last_change":"2024-05-21T17:41:33.206294+0000","last_active":"2024-05-21T17:41:35.504945+0000","last_peered":"2024-05-21T17:41:35.504945+0000","last_clean":"2024-05-21T17:41:35.504945+0000","last_became_active":"2024-05-21T17:41:33.205960+0000","last_became_peered":"2024-05-21T17:41:33.205960+0000","last_unstale":"2024-05-21T17:41:35.504945+0000","last_undegraded":"2024-05-21T17:41:35.504945+0000","last_fullsized":"2024-05-21T17:41:35.504945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:47:06.686603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504088+0000","last_change":"2024-05-21T17:41:32.198839+0000","last_active":"2024-05-21T17:41:35.504088+0000","last_peered":"2024-05-21T17:41:35.504088+0000","last_clean":"2024-05-21T17:41:35.504088+0000","last_became_active":"2024-05-21T17:41:32.198523+0000","last_became_peered":"2024-05-21T17:41:32.198523+0000","last_unstale":"2024-05-21T17:41:35.504088+0000","last_undegraded":"2024-05-21T17:41:35.504088+0000","last_fullsized":"2024-05-21T17:41:35.504088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:07:05.404785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.197963+0000","last_change":"2024-05-21T17:41:33.186217+0000","last_active":"2024-05-21T17:41:34.197963+0000","last_peered":"2024-05-21T17:41:34.197963+0000","last_clean":"2024-05-21T17:41:34.197963+0000","last_became_active":"2024-05-21T17:41:33.185935+0000","last_became_peered":"2024-05-21T17:41:33.185935+0000","last_unstale":"2024-05-21T17:41:34.197963+0000","last_undegraded":"2024-05-21T17:41:34.197963+0000","last_fullsized":"2024-05-21T17:41:34.197963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:00:42.905885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.195682+0000","last_change":"2024-05-21T17:41:32.191785+0000","last_active":"2024-05-21T17:41:34.195682+0000","last_peered":"2024-05-21T17:41:34.195682+0000","last_clean":"2024-05-21T17:41:34.195682+0000","last_became_active":"2024-05-21T17:41:32.190999+0000","last_became_peered":"2024-05-21T17:41:32.190999+0000","last_unstale":"2024-05-21T17:41:34.195682+0000","last_undegraded":"2024-05-21T17:41:34.195682+0000","last_fullsized":"2024-05-21T17:41:34.195682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:06:52.250548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504863+0000","last_change":"2024-05-21T17:41:33.213982+0000","last_active":"2024-05-21T17:41:35.504863+0000","last_peered":"2024-05-21T17:41:35.504863+0000","last_clean":"2024-05-21T17:41:35.504863+0000","last_became_active":"2024-05-21T17:41:33.213225+0000","last_became_peered":"2024-05-21T17:41:33.213225+0000","last_unstale":"2024-05-21T17:41:35.504863+0000","last_undegraded":"2024-05-21T17:41:35.504863+0000","last_fullsized":"2024-05-21T17:41:35.504863+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:07:15.129363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196466+0000","last_change":"2024-05-21T17:41:32.182693+0000","last_active":"2024-05-21T17:41:34.196466+0000","last_peered":"2024-05-21T17:41:34.196466+0000","last_clean":"2024-05-21T17:41:34.196466+0000","last_became_active":"2024-05-21T17:41:32.182420+0000","last_became_peered":"2024-05-21T17:41:32.182420+0000","last_unstale":"2024-05-21T17:41:34.196466+0000","last_undegraded":"2024-05-21T17:41:34.196466+0000","last_fullsized":"2024-05-21T17:41:34.196466+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:26:32.625845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198167+0000","last_change":"2024-05-21T17:41:33.198020+0000","last_active":"2024-05-21T17:41:34.198167+0000","last_peered":"2024-05-21T17:41:34.198167+0000","last_clean":"2024-05-21T17:41:34.198167+0000","last_became_active":"2024-05-21T17:41:33.197187+0000","last_became_peered":"2024-05-21T17:41:33.197187+0000","last_unstale":"2024-05-21T17:41:34.198167+0000","last_undegraded":"2024-05-21T17:41:34.198167+0000","last_fullsized":"2024-05-21T17:41:34.198167+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T23:37:09.553771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504321+0000","last_change":"2024-05-21T17:41:32.197952+0000","last_active":"2024-05-21T17:41:35.504321+0000","last_peered":"2024-05-21T17:41:35.504321+0000","last_clean":"2024-05-21T17:41:35.504321+0000","last_became_active":"2024-05-21T17:41:32.197316+0000","last_became_peered":"2024-05-21T17:41:32.197316+0000","last_unstale":"2024-05-21T17:41:35.504321+0000","last_undegraded":"2024-05-21T17:41:35.504321+0000","last_fullsized":"2024-05-21T17:41:35.504321+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T18:04:40.497094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181105+0000","last_change":"2024-05-21T17:41:33.196406+0000","last_active":"2024-05-21T17:41:35.181105+0000","last_peered":"2024-05-21T17:41:35.181105+0000","last_clean":"2024-05-21T17:41:35.181105+0000","last_became_active":"2024-05-21T17:41:33.195986+0000","last_became_peered":"2024-05-21T17:41:33.195986+0000","last_unstale":"2024-05-21T17:41:35.181105+0000","last_undegraded":"2024-05-21T17:41:35.181105+0000","last_fullsized":"2024-05-21T17:41:35.181105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T03:54:27.905968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196455+0000","last_change":"2024-05-21T17:41:32.192119+0000","last_active":"2024-05-21T17:41:34.196455+0000","last_peered":"2024-05-21T17:41:34.196455+0000","last_clean":"2024-05-21T17:41:34.196455+0000","last_became_active":"2024-05-21T17:41:32.191275+0000","last_became_peered":"2024-05-21T17:41:32.191275+0000","last_unstale":"2024-05-21T17:41:34.196455+0000","last_undegraded":"2024-05-21T17:41:34.196455+0000","last_fullsized":"2024-05-21T17:41:34.196455+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T01:58:39.302037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.181217+0000","last_change":"2024-05-21T17:41:33.196519+0000","last_active":"2024-05-21T17:41:35.181217+0000","last_peered":"2024-05-21T17:41:35.181217+0000","last_clean":"2024-05-21T17:41:35.181217+0000","last_became_active":"2024-05-21T17:41:33.194635+0000","last_became_peered":"2024-05-21T17:41:33.194635+0000","last_unstale":"2024-05-21T17:41:35.181217+0000","last_undegraded":"2024-05-21T17:41:35.181217+0000","last_fullsized":"2024-05-21T17:41:35.181217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:00:42.905885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"15'2","reported_seq":16,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.505022+0000","last_change":"2024-05-21T17:41:32.198015+0000","last_active":"2024-05-21T17:41:35.505022+0000","last_peered":"2024-05-21T17:41:35.505022+0000","last_clean":"2024-05-21T17:41:35.505022+0000","last_became_active":"2024-05-21T17:41:32.197483+0000","last_became_peered":"2024-05-21T17:41:32.197483+0000","last_unstale":"2024-05-21T17:41:35.505022+0000","last_undegraded":"2024-05-21T17:41:35.505022+0000","last_fullsized":"2024-05-21T17:41:35.505022+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:00:41.900604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.504078+0000","last_change":"2024-05-21T17:41:33.207480+0000","last_active":"2024-05-21T17:41:35.504078+0000","last_peered":"2024-05-21T17:41:35.504078+0000","last_clean":"2024-05-21T17:41:35.504078+0000","last_became_active":"2024-05-21T17:41:33.207211+0000","last_became_peered":"2024-05-21T17:41:33.207211+0000","last_unstale":"2024-05-21T17:41:35.504078+0000","last_undegraded":"2024-05-21T17:41:35.504078+0000","last_fullsized":"2024-05-21T17:41:35.504078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-23T04:55:50.762859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"15'1","reported_seq":15,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.198168+0000","last_change":"2024-05-21T17:41:32.180196+0000","last_active":"2024-05-21T17:41:34.198168+0000","last_peered":"2024-05-21T17:41:34.198168+0000","last_clean":"2024-05-21T17:41:34.198168+0000","last_became_active":"2024-05-21T17:41:32.179887+0000","last_became_peered":"2024-05-21T17:41:32.179887+0000","last_unstale":"2024-05-21T17:41:34.198168+0000","last_undegraded":"2024-05-21T17:41:34.198168+0000","last_fullsized":"2024-05-21T17:41:34.198168+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:06:50.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:35.180828+0000","last_change":"2024-05-21T17:41:33.196424+0000","last_active":"2024-05-21T17:41:35.180828+0000","last_peered":"2024-05-21T17:41:35.180828+0000","last_clean":"2024-05-21T17:41:35.180828+0000","last_became_active":"2024-05-21T17:41:33.196119+0000","last_became_peered":"2024-05-21T17:41:33.196119+0000","last_unstale":"2024-05-21T17:41:35.180828+0000","last_undegraded":"2024-05-21T17:41:35.180828+0000","last_fullsized":"2024-05-21T17:41:35.180828+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:32.164627+0000","last_clean_scrub_stamp":"2024-05-21T17:41:32.164627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T20:29:26.312866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"15'4","reported_seq":18,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-21T17:41:34.196681+0000","last_change":"2024-05-21T17:41:32.182802+0000","last_active":"2024-05-21T17:41:34.196681+0000","last_peered":"2024-05-21T17:41:34.196681+0000","last_clean":"2024-05-21T17:41:34.196681+0000","last_became_active":"2024-05-21T17:41:32.182446+0000","last_became_peered":"2024-05-21T17:41:32.182446+0000","last_unstale":"2024-05-21T17:41:34.196681+0000","last_undegraded":"2024-05-21T17:41:34.196681+0000","last_fullsized":"2024-05-21T17:41:34.196681+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-21T17:41:31.159346+0000","last_clean_scrub_stamp":"2024-05-21T17:41:31.159346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-22T19:06:50.953546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17264,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":176128,"data_stored":52460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":642780,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705670,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":641696,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":51236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8632,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-21T17:41:41.463 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-21T17:41:41.464 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-21T17:41:41.464 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-21T17:41:41.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.601+0000 7f4c21342640 1 Processor -- start 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.601+0000 7f4c21342640 1 -- start start 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c21342640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c075540 0x7f4c1c0a6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c21342640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4c1c0a1570 0x7f4c1c0a9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c21342640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c1c0a6c90 con 0x7f4c1c0a9690 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c21342640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4c1c075920 con 0x7f4c1c0a1570 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c21342640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1c146400 con 0x7f4c1c075540 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c1affd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c075540 0x7f4c1c0a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c1b7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4c1c0a9690 0x7f4c1c145830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60748/0 (socket says 172.21.15.3:60748) 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c1b7fe640 1 -- 172.21.15.3:0/3928807072 learned_addr learned my addr 172.21.15.3:0/3928807072 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:41.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c1affd640 1 -- 172.21.15.3:0/3928807072 >> v2:172.21.15.203:3300/0 conn(0x7f4c1c0a1570 msgr2=0x7f4c1c0a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:41.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c1a7fc640 1 --2- 172.21.15.3:0/3928807072 >> v2:172.21.15.203:3300/0 conn(0x7f4c1c0a1570 0x7f4c1c0a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c19ffb640 1 -- 172.21.15.3:0/3928807072 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 450172318 0 0) 0x7f4c1c0a6c90 con 0x7f4c1c0a9690 2024-05-21T17:41:41.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c1affd640 1 --2- 172.21.15.3:0/3928807072 >> v2:172.21.15.203:3300/0 conn(0x7f4c1c0a1570 0x7f4c1c0a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c1affd640 1 -- 172.21.15.3:0/3928807072 >> v1:172.21.15.3:6789/0 conn(0x7f4c1c0a9690 legacy=0x7f4c1c145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c1affd640 1 -- 172.21.15.3:0/3928807072 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1c1466c0 con 0x7f4c1c075540 2024-05-21T17:41:41.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.602+0000 7f4c1a7fc640 1 --2- 172.21.15.3:0/3928807072 >> v2:172.21.15.203:3300/0 conn(0x7f4c1c0a1570 0x7f4c1c0a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.603+0000 7f4c1affd640 1 --2- 172.21.15.3:0/3928807072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c075540 0x7f4c1c0a6750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009d90 tx=0x7f4c0c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=249a1924ef69dc8a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.603+0000 7f4c19ffb640 1 -- 172.21.15.3:0/3928807072 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c039070 con 0x7f4c1c075540 2024-05-21T17:41:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.603+0000 7f4c19ffb640 1 -- 172.21.15.3:0/3928807072 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c0c02cbc0 con 0x7f4c1c075540 2024-05-21T17:41:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.603+0000 7f4c19ffb640 1 -- 172.21.15.3:0/3928807072 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c034440 con 0x7f4c1c075540 2024-05-21T17:41:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.604+0000 7f4c21342640 1 -- 172.21.15.3:0/3928807072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c075540 msgr2=0x7f4c1c0a6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.604+0000 7f4c21342640 1 --2- 172.21.15.3:0/3928807072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c075540 0x7f4c1c0a6750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009d90 tx=0x7f4c0c02be10 comp rx=0 tx=0).stop 2024-05-21T17:41:41.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.604+0000 7f4c21342640 1 -- 172.21.15.3:0/3928807072 shutdown_connections 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.604+0000 7f4c21342640 1 --2- 172.21.15.3:0/3928807072 >> v2:172.21.15.203:3300/0 conn(0x7f4c1c0a1570 0x7f4c1c0a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.604+0000 7f4c21342640 1 --2- 172.21.15.3:0/3928807072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c075540 0x7f4c1c0a6750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.604+0000 7f4c21342640 1 -- 172.21.15.3:0/3928807072 >> 172.21.15.3:0/3928807072 conn(0x7f4c1c0b8050 msgr2=0x7f4c1c0b8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.604+0000 7f4c21342640 1 -- 172.21.15.3:0/3928807072 shutdown_connections 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.604+0000 7f4c21342640 1 -- 172.21.15.3:0/3928807072 wait complete. 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.605+0000 7f4c21342640 1 Processor -- start 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.605+0000 7f4c21342640 1 -- start start 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c21342640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c1c075540 0x7f4c1c1459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c21342640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c0a1570 0x7f4c1c144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c21342640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c1c0a9690 0x7f4c1c13e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c21342640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c150a00 con 0x7f4c1c0a9690 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c21342640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1c146190 con 0x7f4c1c075540 2024-05-21T17:41:41.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c21342640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1c150b80 con 0x7f4c1c0a1570 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1a7fc640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c0a1570 0x7f4c1c144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c1c0a9690 0x7f4c1c13e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1a7fc640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c0a1570 0x7f4c1c144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41130/0 (socket says 172.21.15.3:41130) 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c1c0a9690 0x7f4c1c13e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51618/0 (socket says 172.21.15.3:51618) 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1a7fc640 1 -- 172.21.15.3:0/2987149663 learned_addr learned my addr 172.21.15.3:0/2987149663 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1affd640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c1c075540 0x7f4c1c1459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1a7fc640 1 -- 172.21.15.3:0/2987149663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c1c075540 msgr2=0x7f4c1c1459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1a7fc640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c1c075540 0x7f4c1c1459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1a7fc640 1 -- 172.21.15.3:0/2987149663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c1c0a9690 msgr2=0x7f4c1c13e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1a7fc640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c1c0a9690 0x7f4c1c13e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1a7fc640 1 -- 172.21.15.3:0/2987149663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1c0a1fd0 con 0x7f4c1c0a1570 2024-05-21T17:41:41.610 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c1a7fc640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c0a1570 0x7f4c1c144820 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000c7f0 tx=0x7f4c1000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:41.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c037fe640 1 -- 172.21.15.3:0/2987149663 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10004230 con 0x7f4c1c0a1570 2024-05-21T17:41:41.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c037fe640 1 -- 172.21.15.3:0/2987149663 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c100043d0 con 0x7f4c1c0a1570 2024-05-21T17:41:41.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c21342640 1 -- 172.21.15.3:0/2987149663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1c143310 con 0x7f4c1c0a1570 2024-05-21T17:41:41.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.606+0000 7f4c037fe640 1 -- 172.21.15.3:0/2987149663 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10007a70 con 0x7f4c1c0a1570 2024-05-21T17:41:41.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.607+0000 7f4c21342640 1 -- 172.21.15.3:0/2987149663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c1c13eb20 con 0x7f4c1c0a1570 2024-05-21T17:41:41.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.607+0000 7f4c21342640 1 -- 172.21.15.3:0/2987149663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c1c0a4df0 con 0x7f4c1c0a1570 2024-05-21T17:41:41.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.609+0000 7f4c037fe640 1 -- 172.21.15.3:0/2987149663 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f4c10007c10 con 0x7f4c1c0a1570 2024-05-21T17:41:41.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.609+0000 7f4c037fe640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:41.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.609+0000 7f4c037fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:41.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.609+0000 7f4c037fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:41.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.609+0000 7f4c037fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:41.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.609+0000 7f4c037fe640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c080af260 0x7f4c080b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.610+0000 7f4c037fe640 1 -- 172.21.15.3:0/2987149663 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f4c100dc930 con 0x7f4c1c0a1570 2024-05-21T17:41:41.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.610+0000 7f4c1affd640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c080af260 0x7f4c080b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.610+0000 7f4c1affd640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c080af260 0x7f4c080b1720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c036000 tx=0x7f4c0c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:41.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.613+0000 7f4c037fe640 1 -- 172.21.15.3:0/2987149663 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4c1c0a4df0 con 0x7f4c1c0a1570 2024-05-21T17:41:41.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.841+0000 7f4c21342640 1 -- 172.21.15.3:0/2987149663 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4c1c0a6c80 con 0x7f4c1c0a1570 2024-05-21T17:41:41.842 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.841+0000 7f4c037fe640 1 -- 172.21.15.3:0/2987149663 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4c1c0a6c80 con 0x7f4c1c0a1570 2024-05-21T17:41:41.843 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:41.843 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-21T17:41:41.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 10 mgrc shutdown 2024-05-21T17:41:41.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 1 -- 172.21.15.3:0/2987149663 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c080af260 msgr2=0x7f4c080b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c080af260 0x7f4c080b1720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c036000 tx=0x7f4c0c030000 comp rx=0 tx=0).stop 2024-05-21T17:41:41.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 1 -- 172.21.15.3:0/2987149663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c0a1570 msgr2=0x7f4c1c144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c0a1570 0x7f4c1c144820 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000c7f0 tx=0x7f4c1000ccb0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 1 -- 172.21.15.3:0/2987149663 shutdown_connections 2024-05-21T17:41:41.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f4c080af260 0x7f4c080b1720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4c1c0a9690 0x7f4c1c13e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4c1c0a1570 0x7f4c1c144820 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 1 --2- 172.21.15.3:0/2987149663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c1c075540 0x7f4c1c1459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.843+0000 7f4c21342640 1 -- 172.21.15.3:0/2987149663 >> 172.21.15.3:0/2987149663 conn(0x7f4c1c0b8050 msgr2=0x7f4c1c0b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.844+0000 7f4c21342640 1 -- 172.21.15.3:0/2987149663 shutdown_connections 2024-05-21T17:41:41.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.844+0000 7f4c21342640 1 -- 172.21.15.3:0/2987149663 wait complete. 2024-05-21T17:41:41.856 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-21T17:41:41.856 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T17:41:41.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.958+0000 7f33bd45e640 1 Processor -- start 2024-05-21T17:41:41.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.958+0000 7f33bd45e640 1 -- start start 2024-05-21T17:41:41.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.958+0000 7f33bd45e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 0x7f33b80a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.958+0000 7f33bd45e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33b80a8220 0x7f33b80adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.958+0000 7f33bd45e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b80a8fe0 con 0x7f33b80a53f0 2024-05-21T17:41:41.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.958+0000 7f33bd45e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f33b80a92a0 con 0x7f33b80a8220 2024-05-21T17:41:41.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.958+0000 7f33bd45e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f33b806ed60 con 0x7f33b80ac570 2024-05-21T17:41:41.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.959+0000 7f33b6ffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 0x7f33b80a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.959+0000 7f33b77fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f33b80a8220 0x7f33b80adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.959+0000 7f33affff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f33b80a53f0 0x7f33b80a7b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60762/0 (socket says 172.21.15.3:60762) 2024-05-21T17:41:41.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.959+0000 7f33affff640 1 -- 172.21.15.3:0/2597832504 learned_addr learned my addr 172.21.15.3:0/2597832504 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:41.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.959+0000 7f33b77fe640 1 -- 172.21.15.3:0/2597832504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 msgr2=0x7f33b80a8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:41.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.959+0000 7f33b77fe640 1 --2- 172.21.15.3:0/2597832504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 0x7f33b80a8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.959+0000 7f33b77fe640 1 -- 172.21.15.3:0/2597832504 >> v1:172.21.15.3:6789/0 conn(0x7f33b80a53f0 legacy=0x7f33b80a7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.959+0000 7f33b77fe640 1 -- 172.21.15.3:0/2597832504 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b80ac950 con 0x7f33b80a8220 2024-05-21T17:41:41.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.959+0000 7f33b6ffd640 1 --2- 172.21.15.3:0/2597832504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 0x7f33b80a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:41.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.960+0000 7f33b77fe640 1 --2- 172.21.15.3:0/2597832504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a8220 0x7f33b80adc90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009d90 tx=0x7f33b002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0a7ec2ede7c404 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:41.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.960+0000 7f33b67fc640 1 -- 172.21.15.3:0/2597832504 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0039070 con 0x7f33b80a8220 2024-05-21T17:41:41.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.960+0000 7f33b67fc640 1 -- 172.21.15.3:0/2597832504 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b002cbc0 con 0x7f33b80a8220 2024-05-21T17:41:41.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.960+0000 7f33b67fc640 1 -- 172.21.15.3:0/2597832504 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0034440 con 0x7f33b80a8220 2024-05-21T17:41:41.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.960+0000 7f33bd45e640 1 -- 172.21.15.3:0/2597832504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a8220 msgr2=0x7f33b80adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.960+0000 7f33bd45e640 1 --2- 172.21.15.3:0/2597832504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a8220 0x7f33b80adc90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009d90 tx=0x7f33b002be10 comp rx=0 tx=0).stop 2024-05-21T17:41:41.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.960+0000 7f33bd45e640 1 -- 172.21.15.3:0/2597832504 shutdown_connections 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.961+0000 7f33bd45e640 1 --2- 172.21.15.3:0/2597832504 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a8220 0x7f33b80adc90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.961+0000 7f33bd45e640 1 --2- 172.21.15.3:0/2597832504 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 0x7f33b80a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.961+0000 7f33bd45e640 1 -- 172.21.15.3:0/2597832504 >> 172.21.15.3:0/2597832504 conn(0x7f33b80bb9e0 msgr2=0x7f33b80bbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.961+0000 7f33bd45e640 1 -- 172.21.15.3:0/2597832504 shutdown_connections 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.961+0000 7f33bd45e640 1 -- 172.21.15.3:0/2597832504 wait complete. 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33bd45e640 1 Processor -- start 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33bd45e640 1 -- start start 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33bd45e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a53f0 0x7f33b81460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33bd45e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b80a8220 0x7f33b814f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33bd45e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 0x7f33b814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.967 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33bd45e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f33b80ac180 con 0x7f33b80a53f0 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33bd45e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f33b80ac000 con 0x7f33b80a8220 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33bd45e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f33b8150bd0 con 0x7f33b80ac570 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33b6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a53f0 0x7f33b81460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33b77fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 0x7f33b814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33b6ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a53f0 0x7f33b81460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51646/0 (socket says 172.21.15.3:51646) 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33b6ffd640 1 -- 172.21.15.3:0/4041545760 learned_addr learned my addr 172.21.15.3:0/4041545760 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.962+0000 7f33affff640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b80a8220 0x7f33b814f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33b77fe640 1 -- 172.21.15.3:0/4041545760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b80a8220 msgr2=0x7f33b814f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33b77fe640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b80a8220 0x7f33b814f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33b77fe640 1 -- 172.21.15.3:0/4041545760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a53f0 msgr2=0x7f33b81460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:41.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33b77fe640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a53f0 0x7f33b81460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:41.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33b77fe640 1 -- 172.21.15.3:0/4041545760 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b81504d0 con 0x7f33b80ac570 2024-05-21T17:41:41.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33b6ffd640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a53f0 0x7f33b81460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:41.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33b77fe640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 0x7f33b814df20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33b0033000 tx=0x7f33b0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:41.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33af7fe640 1 -- 172.21.15.3:0/4041545760 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0039050 con 0x7f33b80ac570 2024-05-21T17:41:41.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33af7fe640 1 -- 172.21.15.3:0/4041545760 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b0045070 con 0x7f33b80ac570 2024-05-21T17:41:41.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33bd45e640 1 -- 172.21.15.3:0/4041545760 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b8146610 con 0x7f33b80ac570 2024-05-21T17:41:41.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33af7fe640 1 -- 172.21.15.3:0/4041545760 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0009450 con 0x7f33b80ac570 2024-05-21T17:41:41.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.963+0000 7f33bd45e640 1 -- 172.21.15.3:0/4041545760 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b813f420 con 0x7f33b80ac570 2024-05-21T17:41:41.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.964+0000 7f33bd45e640 1 -- 172.21.15.3:0/4041545760 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b8069a90 con 0x7f33b80ac570 2024-05-21T17:41:41.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.965+0000 7f33af7fe640 1 -- 172.21.15.3:0/4041545760 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f33b003e850 con 0x7f33b80ac570 2024-05-21T17:41:41.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.965+0000 7f33af7fe640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:41.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.966+0000 7f33af7fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:41.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.966+0000 7f33af7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:41.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.966+0000 7f33af7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:41.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.966+0000 7f33af7fe640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f33980af260 0x7f33980b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:41.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.966+0000 7f33af7fe640 1 -- 172.21.15.3:0/4041545760 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f33b00fff30 con 0x7f33b80ac570 2024-05-21T17:41:41.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.966+0000 7f33b6ffd640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f33980af260 0x7f33980b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:41.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.966+0000 7f33b6ffd640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f33980af260 0x7f33980b1720 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f33a80095e0 tx=0x7f33a8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:41.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:41.969+0000 7f33af7fe640 1 -- 172.21.15.3:0/4041545760 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f33b8146610 con 0x7f33b80ac570 2024-05-21T17:41:42.198 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.198+0000 7f33bd45e640 1 -- 172.21.15.3:0/4041545760 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33b8075c20 con 0x7f33b80ac570 2024-05-21T17:41:42.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.198+0000 7f33af7fe640 1 -- 172.21.15.3:0/4041545760 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+4345 (secure 0 0 0) 0x7f33b81504d0 con 0x7f33b80ac570 2024-05-21T17:41:42.199 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:42.200 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/3741447020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3741447020},{"type":"v1","addr":"172.21.15.203:6827","nonce":3741447020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6810/205822275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":205822275},{"type":"v1","addr":"172.21.15.3:6810","nonce":205822275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-21T17:41:32.553094+0000","modified":"2024-05-21T17:41:35.579599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6811/1903526925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6809","nonce":1903526925},{"type":"v1","addr":"172.21.15.3:6811","nonce":1903526925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-05-21T17:41:42.200 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-21T17:41:42.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 10 mgrc shutdown 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 -- 172.21.15.3:0/4041545760 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f33980af260 msgr2=0x7f33980b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f33980af260 0x7f33980b1720 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f33a80095e0 tx=0x7f33a8009190 comp rx=0 tx=0).stop 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 -- 172.21.15.3:0/4041545760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 msgr2=0x7f33b814df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 0x7f33b814df20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33b0033000 tx=0x7f33b0004a40 comp rx=0 tx=0).stop 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 -- 172.21.15.3:0/4041545760 shutdown_connections 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f33980af260 0x7f33980b1720 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33b80ac570 0x7f33b814df20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33b80a8220 0x7f33b814f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 --2- 172.21.15.3:0/4041545760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33b80a53f0 0x7f33b81460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 -- 172.21.15.3:0/4041545760 >> 172.21.15.3:0/4041545760 conn(0x7f33b80bb9e0 msgr2=0x7f33b806cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:42.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 -- 172.21.15.3:0/4041545760 shutdown_connections 2024-05-21T17:41:42.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.200+0000 7f33bd45e640 1 -- 172.21.15.3:0/4041545760 wait complete. 2024-05-21T17:41:42.213 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T17:41:42.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.315+0000 7f37f792b640 1 Processor -- start 2024-05-21T17:41:42.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.315+0000 7f37f792b640 1 -- start start 2024-05-21T17:41:42.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.316+0000 7f37f792b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37f00ac330 0x7f37f00ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.316+0000 7f37f792b640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 0x7f37f0150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.316+0000 7f37f792b640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f0146b20 con 0x7f37f0149720 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.316+0000 7f37f792b640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f37f00a8530 con 0x7f37f00ac330 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.316+0000 7f37f792b640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f37f0146de0 con 0x7f37f0146650 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.316+0000 7f37f56c7640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f37f00ac330 0x7f37f00ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.316+0000 7f37f5ec8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 0x7f37f0150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.316+0000 7f37f4ec6640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f37f0149720 0x7f37f0149b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60766/0 (socket says 172.21.15.3:60766) 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.316+0000 7f37f4ec6640 1 -- 172.21.15.3:0/2180778825 learned_addr learned my addr 172.21.15.3:0/2180778825 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.317+0000 7f37f56c7640 1 -- 172.21.15.3:0/2180778825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 msgr2=0x7f37f0150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.317+0000 7f37f56c7640 1 --2- 172.21.15.3:0/2180778825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 0x7f37f0150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.317+0000 7f37f56c7640 1 -- 172.21.15.3:0/2180778825 >> v1:172.21.15.3:6789/0 conn(0x7f37f0149720 legacy=0x7f37f0149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.317+0000 7f37f56c7640 1 -- 172.21.15.3:0/2180778825 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f00ac710 con 0x7f37f00ac330 2024-05-21T17:41:42.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.317+0000 7f37f5ec8640 1 --2- 172.21.15.3:0/2180778825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 0x7f37f0150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:42.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.317+0000 7f37f56c7640 1 --2- 172.21.15.3:0/2180778825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 0x7f37f00ad480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0097a0 tx=0x7f37ec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dea9b4e1387ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:42.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.317+0000 7f37e7fff640 1 -- 172.21.15.3:0/2180778825 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0315a0 con 0x7f37f00ac330 2024-05-21T17:41:42.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.317+0000 7f37e7fff640 1 -- 172.21.15.3:0/2180778825 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ec037070 con 0x7f37f00ac330 2024-05-21T17:41:42.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.317+0000 7f37e7fff640 1 -- 172.21.15.3:0/2180778825 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec031960 con 0x7f37f00ac330 2024-05-21T17:41:42.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.318+0000 7f37f792b640 1 -- 172.21.15.3:0/2180778825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 msgr2=0x7f37f00ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.318+0000 7f37f792b640 1 --2- 172.21.15.3:0/2180778825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 0x7f37f00ad480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0097a0 tx=0x7f37ec02b840 comp rx=0 tx=0).stop 2024-05-21T17:41:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.318+0000 7f37f792b640 1 -- 172.21.15.3:0/2180778825 shutdown_connections 2024-05-21T17:41:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.318+0000 7f37f792b640 1 --2- 172.21.15.3:0/2180778825 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 0x7f37f0150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.318+0000 7f37f792b640 1 --2- 172.21.15.3:0/2180778825 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 0x7f37f00ad480 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.318+0000 7f37f792b640 1 -- 172.21.15.3:0/2180778825 >> 172.21.15.3:0/2180778825 conn(0x7f37f00aab00 msgr2=0x7f37f00aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.318+0000 7f37f792b640 1 -- 172.21.15.3:0/2180778825 shutdown_connections 2024-05-21T17:41:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.318+0000 7f37f792b640 1 -- 172.21.15.3:0/2180778825 wait complete. 2024-05-21T17:41:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.319+0000 7f37f792b640 1 Processor -- start 2024-05-21T17:41:42.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.319+0000 7f37f792b640 1 -- start start 2024-05-21T17:41:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f792b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 0x7f37f01db640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f792b640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 0x7f37f01dc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f792b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f0149720 0x7f37f01dc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f792b640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f37f00adec0 con 0x7f37f00ac330 2024-05-21T17:41:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f792b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f37f006cf00 con 0x7f37f0149720 2024-05-21T17:41:42.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f792b640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f37f014a210 con 0x7f37f0146650 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f56c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 0x7f37f01db640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f56c7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 0x7f37f01db640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51676/0 (socket says 172.21.15.3:51676) 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f56c7640 1 -- 172.21.15.3:0/1167082446 learned_addr learned my addr 172.21.15.3:0/1167082446 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f4ec6640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 0x7f37f01dc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f56c7640 1 -- 172.21.15.3:0/1167082446 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 msgr2=0x7f37f01dc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f56c7640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 0x7f37f01dc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f5ec8640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f0149720 0x7f37f01dc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f56c7640 1 -- 172.21.15.3:0/1167082446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f0149720 msgr2=0x7f37f01dc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f56c7640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f0149720 0x7f37f01dc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f56c7640 1 -- 172.21.15.3:0/1167082446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f0143190 con 0x7f37f00ac330 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.320+0000 7f37f56c7640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 0x7f37f01db640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0098c0 tx=0x7f37ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.321+0000 7f37e5ffb640 1 -- 172.21.15.3:0/1167082446 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec032630 con 0x7f37f00ac330 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.321+0000 7f37e5ffb640 1 -- 172.21.15.3:0/1167082446 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ec0327d0 con 0x7f37f00ac330 2024-05-21T17:41:42.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.321+0000 7f37f792b640 1 -- 172.21.15.3:0/1167082446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f01dcd00 con 0x7f37f00ac330 2024-05-21T17:41:42.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.321+0000 7f37e5ffb640 1 -- 172.21.15.3:0/1167082446 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec032bc0 con 0x7f37f00ac330 2024-05-21T17:41:42.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.321+0000 7f37f792b640 1 -- 172.21.15.3:0/1167082446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f0143940 con 0x7f37f00ac330 2024-05-21T17:41:42.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.322+0000 7f37f792b640 1 -- 172.21.15.3:0/1167082446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f0069a90 con 0x7f37f00ac330 2024-05-21T17:41:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.323+0000 7f37e5ffb640 1 -- 172.21.15.3:0/1167082446 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f37ec04e430 con 0x7f37f00ac330 2024-05-21T17:41:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.323+0000 7f37e5ffb640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.324+0000 7f37e5ffb640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.324+0000 7f37e5ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.324+0000 7f37e5ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.324+0000 7f37e5ffb640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37d40af2e0 0x7f37d40b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.324+0000 7f37e5ffb640 1 -- 172.21.15.3:0/1167082446 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f37ec02f030 con 0x7f37f00ac330 2024-05-21T17:41:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.324+0000 7f37f4ec6640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37d40af2e0 0x7f37d40b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.326 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.325+0000 7f37f4ec6640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37d40af2e0 0x7f37d40b17a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f37f01406a0 tx=0x7f37e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:42.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.327+0000 7f37e5ffb640 1 -- 172.21.15.3:0/1167082446 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f37ec040070 con 0x7f37f00ac330 2024-05-21T17:41:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.558+0000 7f37f792b640 1 -- 172.21.15.3:0/1167082446 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37f00a7d70 con 0x7f37f00ac330 2024-05-21T17:41:42.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.558+0000 7f37e5ffb640 1 -- 172.21.15.3:0/1167082446 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+4345 (secure 0 0 0) 0x7f37ec0c86e0 con 0x7f37f00ac330 2024-05-21T17:41:42.560 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:42.560 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/3741447020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3741447020},{"type":"v1","addr":"172.21.15.203:6827","nonce":3741447020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6810/205822275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":205822275},{"type":"v1","addr":"172.21.15.3:6810","nonce":205822275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-21T17:41:32.553094+0000","modified":"2024-05-21T17:41:35.579599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6811/1903526925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6809","nonce":1903526925},{"type":"v1","addr":"172.21.15.3:6811","nonce":1903526925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-05-21T17:41:42.560 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-21T17:41:42.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 10 mgrc shutdown 2024-05-21T17:41:42.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 -- 172.21.15.3:0/1167082446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37d40af2e0 msgr2=0x7f37d40b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37d40af2e0 0x7f37d40b17a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f37f01406a0 tx=0x7f37e0009040 comp rx=0 tx=0).stop 2024-05-21T17:41:42.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 -- 172.21.15.3:0/1167082446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 msgr2=0x7f37f01db640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 0x7f37f01db640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0098c0 tx=0x7f37ec0046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 -- 172.21.15.3:0/1167082446 shutdown_connections 2024-05-21T17:41:42.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f37d40af2e0 0x7f37d40b17a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37f0149720 0x7f37f01dc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f37f0146650 0x7f37f01dc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 --2- 172.21.15.3:0/1167082446 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f37f00ac330 0x7f37f01db640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 -- 172.21.15.3:0/1167082446 >> 172.21.15.3:0/1167082446 conn(0x7f37f00aab00 msgr2=0x7f37f00bc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:42.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 -- 172.21.15.3:0/1167082446 shutdown_connections 2024-05-21T17:41:42.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.561+0000 7f37f792b640 1 -- 172.21.15.3:0/1167082446 wait complete. 2024-05-21T17:41:42.573 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T17:41:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.675+0000 7f4207dbf640 1 Processor -- start 2024-05-21T17:41:42.676 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.675+0000 7f4207dbf640 1 -- start start 2024-05-21T17:41:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.675+0000 7f4207dbf640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42000a8a80 0x7f42000a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.675+0000 7f4207dbf640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a4840 0x7f42000a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.675+0000 7f4207dbf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f420006e4d0 con 0x7f4200073e60 2024-05-21T17:41:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.675+0000 7f4207dbf640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f420006e2a0 con 0x7f42000a8a80 2024-05-21T17:41:42.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.675+0000 7f4207dbf640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f42000a8e60 con 0x7f42000a4840 2024-05-21T17:41:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.676+0000 7f4205b5b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42000a8a80 0x7f42000a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.676+0000 7f420635c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a4840 0x7f42000a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.676+0000 7f4205b5b640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f42000a8a80 0x7f42000a1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51684/0 (socket says 172.21.15.3:51684) 2024-05-21T17:41:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.676+0000 7f4205b5b640 1 -- 172.21.15.3:0/2965197655 learned_addr learned my addr 172.21.15.3:0/2965197655 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.676+0000 7f420635c640 1 -- 172.21.15.3:0/2965197655 >> v2:172.21.15.3:3300/0 conn(0x7f42000a8a80 msgr2=0x7f42000a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.676+0000 7f420635c640 1 --2- 172.21.15.3:0/2965197655 >> v2:172.21.15.3:3300/0 conn(0x7f42000a8a80 0x7f42000a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.676+0000 7f420635c640 1 -- 172.21.15.3:0/2965197655 >> v1:172.21.15.203:6789/0 conn(0x7f4200073e60 legacy=0x7f42000a7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.676+0000 7f420635c640 1 -- 172.21.15.3:0/2965197655 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42000a6270 con 0x7f42000a4840 2024-05-21T17:41:42.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.676+0000 7f4205b5b640 1 --2- 172.21.15.3:0/2965197655 >> v2:172.21.15.3:3300/0 conn(0x7f42000a8a80 0x7f42000a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.677+0000 7f420635c640 1 --2- 172.21.15.3:0/2965197655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a4840 0x7f42000a4cf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41f80097a0 tx=0x7f41f802b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de3193da990142b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.677+0000 7f4204b59640 1 -- 172.21.15.3:0/2965197655 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8037070 con 0x7f42000a4840 2024-05-21T17:41:42.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.677+0000 7f4204b59640 1 -- 172.21.15.3:0/2965197655 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f8030710 con 0x7f42000a4840 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.677+0000 7f4204b59640 1 -- 172.21.15.3:0/2965197655 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8030ae0 con 0x7f42000a4840 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.677+0000 7f4207dbf640 1 -- 172.21.15.3:0/2965197655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a4840 msgr2=0x7f42000a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.677+0000 7f4207dbf640 1 --2- 172.21.15.3:0/2965197655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a4840 0x7f42000a4cf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41f80097a0 tx=0x7f41f802b6f0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.677+0000 7f4207dbf640 1 -- 172.21.15.3:0/2965197655 shutdown_connections 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.677+0000 7f4207dbf640 1 --2- 172.21.15.3:0/2965197655 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a4840 0x7f42000a4cf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.677+0000 7f4207dbf640 1 --2- 172.21.15.3:0/2965197655 >> v2:172.21.15.3:3300/0 conn(0x7f42000a8a80 0x7f42000a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.677+0000 7f4207dbf640 1 -- 172.21.15.3:0/2965197655 >> 172.21.15.3:0/2965197655 conn(0x7f420006ccf0 msgr2=0x7f42000a6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.678+0000 7f4207dbf640 1 -- 172.21.15.3:0/2965197655 shutdown_connections 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.678+0000 7f4207dbf640 1 -- 172.21.15.3:0/2965197655 wait complete. 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f4207dbf640 1 Processor -- start 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f4207dbf640 1 -- start start 2024-05-21T17:41:42.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f4207dbf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4200073e60 0x7f42001dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f4207dbf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42000a4840 0x7f42001db890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f4207dbf640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a8a80 0x7f42001d81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f4207dbf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f420006dc20 con 0x7f42000a4840 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f4207dbf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42000a9010 con 0x7f4200073e60 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f4207dbf640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f420006dda0 con 0x7f42000a8a80 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f420535a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42000a4840 0x7f42001db890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f420635c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a8a80 0x7f42001d81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f420635c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a8a80 0x7f42001d81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41164/0 (socket says 172.21.15.3:41164) 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f420635c640 1 -- 172.21.15.3:0/2505197375 learned_addr learned my addr 172.21.15.3:0/2505197375 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f420535a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42000a4840 0x7f42001db890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51688/0 (socket says 172.21.15.3:51688) 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.679+0000 7f4205b5b640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4200073e60 0x7f42001dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f420635c640 1 -- 172.21.15.3:0/2505197375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4200073e60 msgr2=0x7f42001dafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f420635c640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4200073e60 0x7f42001dafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f420635c640 1 -- 172.21.15.3:0/2505197375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42000a4840 msgr2=0x7f42001db890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f420635c640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42000a4840 0x7f42001db890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f420635c640 1 -- 172.21.15.3:0/2505197375 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4200072d10 con 0x7f42000a8a80 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f420535a640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42000a4840 0x7f42001db890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f420635c640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a8a80 0x7f42001d81c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f41f80098c0 tx=0x7f41f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f41fe7fc640 1 -- 172.21.15.3:0/2505197375 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8041020 con 0x7f42000a8a80 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f41fe7fc640 1 -- 172.21.15.3:0/2505197375 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f803c870 con 0x7f42000a8a80 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f4207dbf640 1 -- 172.21.15.3:0/2505197375 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42001d7f00 con 0x7f42000a8a80 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f41fe7fc640 1 -- 172.21.15.3:0/2505197375 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8037040 con 0x7f42000a8a80 2024-05-21T17:41:42.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.680+0000 7f4207dbf640 1 -- 172.21.15.3:0/2505197375 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001dbdd0 con 0x7f42000a8a80 2024-05-21T17:41:42.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.681+0000 7f4207dbf640 1 -- 172.21.15.3:0/2505197375 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42000b6490 con 0x7f42000a8a80 2024-05-21T17:41:42.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.682+0000 7f41fe7fc640 1 -- 172.21.15.3:0/2505197375 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f41f8044020 con 0x7f42000a8a80 2024-05-21T17:41:42.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.682+0000 7f41fe7fc640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.683+0000 7f41fe7fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.683+0000 7f41fe7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.683+0000 7f41fe7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.683+0000 7f41fe7fc640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f41ec0af260 0x7f41ec0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.683+0000 7f41fe7fc640 1 -- 172.21.15.3:0/2505197375 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f41f80fe220 con 0x7f42000a8a80 2024-05-21T17:41:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.683+0000 7f4205b5b640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f41ec0af260 0x7f41ec0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:42.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.684+0000 7f4205b5b640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f41ec0af260 0x7f41ec0b1720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006ca0 tx=0x7f41f000f040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:42.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.687+0000 7f41fe7fc640 1 -- 172.21.15.3:0/2505197375 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f41f804b030 con 0x7f42000a8a80 2024-05-21T17:41:42.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.878+0000 7f4207dbf640 1 -- 172.21.15.3:0/2505197375 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42000a6050 con 0x7f42000a8a80 2024-05-21T17:41:42.880 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.879+0000 7f41fe7fc640 1 -- 172.21.15.3:0/2505197375 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+13521 (secure 0 0 0) 0x7f41f80c7fd0 con 0x7f42000a8a80 2024-05-21T17:41:42.880 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:42.880 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":16,"fsid":"e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442","created":"2024-05-21T17:41:17.034337+0000","modified":"2024-05-21T17:41:34.184816+0000","last_up_change":"2024-05-21T17:41:25.136644+0000","last_in_change":"2024-05-21T17:41:18.179591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T17:41:25.537561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-21T17:41:26.016407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-05-21T17:41:30.560078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-05-21T17:41:31.496059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7ac7071-219d-4cf4-9f8a-977f495fd0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6801","nonce":3296405141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6803","nonce":3296405141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6807","nonce":3296405141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6805","nonce":3296405141}]},"public_addr":"172.21.15.3:6801/3296405141","cluster_addr":"172.21.15.3:6803/3296405141","heartbeat_back_addr":"172.21.15.3:6807/3296405141","heartbeat_front_addr":"172.21.15.3:6805/3296405141","state":["exists","up"]},{"osd":1,"uuid":"89df5105-5900-4ffa-b8c1-6f480865a085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6801","nonce":1777872358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6803","nonce":1777872358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6810","nonce":1777872358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6805","nonce":1777872358}]},"public_addr":"172.21.15.203:6801/1777872358","cluster_addr":"172.21.15.203:6803/1777872358","heartbeat_back_addr":"172.21.15.203:6810/1777872358","heartbeat_front_addr":"172.21.15.203:6805/1777872358","state":["exists","up"]},{"osd":2,"uuid":"15f83334-8efc-40c2-af0c-eeed45923086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6817","nonce":1069390381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6819","nonce":1069390381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6823","nonce":1069390381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6821","nonce":1069390381}]},"public_addr":"172.21.15.203:6817/1069390381","cluster_addr":"172.21.15.203:6819/1069390381","heartbeat_back_addr":"172.21.15.203:6823/1069390381","heartbeat_front_addr":"172.21.15.203:6821/1069390381","state":["exists","up"]},{"osd":3,"uuid":"6a27ac8e-f957-4455-b477-3090081766c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6808","nonce":1072938612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6811","nonce":1072938612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6815","nonce":1072938612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6813","nonce":1072938612}]},"public_addr":"172.21.15.203:6808/1072938612","cluster_addr":"172.21.15.203:6811/1072938612","heartbeat_back_addr":"172.21.15.203:6815/1072938612","heartbeat_front_addr":"172.21.15.203:6813/1072938612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.224040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.344461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.319792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.332782+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T17:41:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.881+0000 7f4207dbf640 10 mgrc shutdown 2024-05-21T17:41:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 -- 172.21.15.3:0/2505197375 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f41ec0af260 msgr2=0x7f41ec0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f41ec0af260 0x7f41ec0b1720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006ca0 tx=0x7f41f000f040 comp rx=0 tx=0).stop 2024-05-21T17:41:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 -- 172.21.15.3:0/2505197375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a8a80 msgr2=0x7f42001d81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a8a80 0x7f42001d81c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f41f80098c0 tx=0x7f41f80046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.883 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 -- 172.21.15.3:0/2505197375 shutdown_connections 2024-05-21T17:41:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f41ec0af260 0x7f41ec0b1720 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f42000a8a80 0x7f42001d81c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f42000a4840 0x7f42001db890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 --2- 172.21.15.3:0/2505197375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4200073e60 0x7f42001dafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 -- 172.21.15.3:0/2505197375 >> 172.21.15.3:0/2505197375 conn(0x7f420006ccf0 msgr2=0x7f42000a7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 -- 172.21.15.3:0/2505197375 shutdown_connections 2024-05-21T17:41:42.884 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.882+0000 7f4207dbf640 1 -- 172.21.15.3:0/2505197375 wait complete. 2024-05-21T17:41:42.894 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T17:41:42.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.996+0000 7fce1fb18640 1 Processor -- start 2024-05-21T17:41:42.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.996+0000 7fce1fb18640 1 -- start start 2024-05-21T17:41:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1fb18640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fce181387f0 0x7fce180ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1fb18640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce18138c80 con 0x7fce180ad6d0 2024-05-21T17:41:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1fb18640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce18138f20 con 0x7fce180abf70 2024-05-21T17:41:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1fb18640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fce18139810 con 0x7fce181387f0 2024-05-21T17:41:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1e0b5640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fce180ad6d0 0x7fce18150b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60774/0 (socket says 172.21.15.3:60774) 2024-05-21T17:41:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1e0b5640 1 -- 172.21.15.3:0/3799178005 learned_addr learned my addr 172.21.15.3:0/3799178005 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1d0b3640 1 --2- 172.21.15.3:0/3799178005 >> v2:172.21.15.203:3300/0 conn(0x7fce181387f0 0x7fce180ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.001 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244929104 0 0) 0x7fce18138c80 con 0x7fce180ad6d0 2024-05-21T17:41:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce04003520 con 0x7fce180ad6d0 2024-05-21T17:41:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1286802842 0 0) 0x7fce04003520 con 0x7fce180ad6d0 2024-05-21T17:41:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 >> v2:172.21.15.203:3300/0 conn(0x7fce181387f0 msgr2=0x7fce180ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 --2- 172.21.15.3:0/3799178005 >> v2:172.21.15.203:3300/0 conn(0x7fce181387f0 0x7fce180ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 >> v1:172.21.15.203:6789/0 conn(0x7fce180abf70 legacy=0x7fce1806c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 --> v1:172.21.15.3:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1813c970 con 0x7fce180ad6d0 2024-05-21T17:41:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 <== mon.0 v1:172.21.15.3:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3544809896 0 0) 0x7fce10001a70 con 0x7fce180ad6d0 2024-05-21T17:41:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 >> v1:172.21.15.3:6789/0 conn(0x7fce180ad6d0 legacy=0x7fce18150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce04003a50 0x7fce04003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce040052e0 0x7fce0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fce18139810 con 0x7fce04004800 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fce0400c1b0 con 0x7fce040052e0 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fce0400c3b0 con 0x7fce04003a50 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1d8b4640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce04003a50 0x7fce04003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.997+0000 7fce1d0b3640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.998+0000 7fce1e0b5640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce040052e0 0x7fce0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.998+0000 7fce1d0b3640 1 -- 172.21.15.3:0/3799178005 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce04003a50 msgr2=0x7fce04003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.998+0000 7fce1d0b3640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce04003a50 0x7fce04003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.998+0000 7fce1d0b3640 1 -- 172.21.15.3:0/3799178005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce040052e0 msgr2=0x7fce0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.998+0000 7fce1d0b3640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce040052e0 0x7fce0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.998+0000 7fce1d0b3640 1 -- 172.21.15.3:0/3799178005 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce04003520 con 0x7fce04004800 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.998+0000 7fce1e0b5640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce040052e0 0x7fce0400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.998+0000 7fce1e0b5640 1 -- 172.21.15.3:0/3799178005 reap_dead start 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1d0b3640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce04004c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce08009a90 tx=0x7fce04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc3d0c76472cc1a server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08039070 con 0x7fce04004800 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce0802c680 con 0x7fce04004800 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1c8b2640 1 -- 172.21.15.3:0/3799178005 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08009690 con 0x7fce04004800 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1fb18640 1 -- 172.21.15.3:0/3799178005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 msgr2=0x7fce04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1fb18640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce04004c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fce08009a90 tx=0x7fce04005830 comp rx=0 tx=0).stop 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1fb18640 1 -- 172.21.15.3:0/3799178005 shutdown_connections 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1fb18640 1 --2- 172.21.15.3:0/3799178005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce04004c80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1fb18640 1 -- 172.21.15.3:0/3799178005 >> 172.21.15.3:0/3799178005 conn(0x7fce180aab70 msgr2=0x7fce180aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1fb18640 1 -- 172.21.15.3:0/3799178005 shutdown_connections 2024-05-21T17:41:43.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:42.999+0000 7fce1fb18640 1 -- 172.21.15.3:0/3799178005 wait complete. 2024-05-21T17:41:43.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.000+0000 7fce1fb18640 1 Processor -- start 2024-05-21T17:41:43.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.000+0000 7fce1fb18640 1 -- start start 2024-05-21T17:41:43.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1fb18640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce18135090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1fb18640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce18146830 0x7fce181dc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1fb18640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce18133870 0x7fce18133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1fb18640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fce180adae0 con 0x7fce04004800 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1fb18640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fce180a52e0 con 0x7fce18133870 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1fb18640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fce180adc60 con 0x7fce18146830 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d0b3640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce18146830 0x7fce181dc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d8b4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce18135090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d0b3640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce18146830 0x7fce181dc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41172/0 (socket says 172.21.15.3:41172) 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d8b4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce18135090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51710/0 (socket says 172.21.15.3:51710) 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d0b3640 1 -- 172.21.15.3:0/4221596474 learned_addr learned my addr 172.21.15.3:0/4221596474 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1e0b5640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce18133870 0x7fce18133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d0b3640 1 -- 172.21.15.3:0/4221596474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce18133870 msgr2=0x7fce18133cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d0b3640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce18133870 0x7fce18133cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d0b3640 1 -- 172.21.15.3:0/4221596474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 msgr2=0x7fce18135090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d0b3640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce18135090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d0b3640 1 -- 172.21.15.3:0/4221596474 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce181dbaf0 con 0x7fce18146830 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.001+0000 7fce1d8b4640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce18135090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.002+0000 7fce1d0b3640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce18146830 0x7fce181dc1f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce08037040 tx=0x7fce08004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.002+0000 7fce0e7fc640 1 -- 172.21.15.3:0/4221596474 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08042020 con 0x7fce18146830 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.002+0000 7fce0e7fc640 1 -- 172.21.15.3:0/4221596474 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce0803a040 con 0x7fce18146830 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.002+0000 7fce1fb18640 1 -- 172.21.15.3:0/4221596474 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce181dca60 con 0x7fce18146830 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.002+0000 7fce0e7fc640 1 -- 172.21.15.3:0/4221596474 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08039070 con 0x7fce18146830 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.002+0000 7fce1fb18640 1 -- 172.21.15.3:0/4221596474 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce180a8390 con 0x7fce18146830 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.003+0000 7fce1fb18640 1 -- 172.21.15.3:0/4221596474 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce18135770 con 0x7fce18146830 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.004+0000 7fce0e7fc640 1 -- 172.21.15.3:0/4221596474 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7fce08045020 con 0x7fce18146830 2024-05-21T17:41:43.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.004+0000 7fce0e7fc640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.005+0000 7fce0e7fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.005+0000 7fce0e7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.005+0000 7fce0e7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.005+0000 7fce0e7fc640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fcdfc0af260 0x7fcdfc0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.005+0000 7fce0e7fc640 1 -- 172.21.15.3:0/4221596474 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7fce080fe7e0 con 0x7fce18146830 2024-05-21T17:41:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.005+0000 7fce1d8b4640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fcdfc0af260 0x7fcdfc0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.006+0000 7fce1d8b4640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fcdfc0af260 0x7fcdfc0b1720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce14001c50 tx=0x7fce14009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:43.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.008+0000 7fce0e7fc640 1 -- 172.21.15.3:0/4221596474 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fce0804c030 con 0x7fce18146830 2024-05-21T17:41:43.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.240+0000 7fce1fb18640 1 -- 172.21.15.3:0/4221596474 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce18069a90 con 0x7fce18146830 2024-05-21T17:41:43.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.240+0000 7fce0e7fc640 1 -- 172.21.15.3:0/4221596474 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+4345 (secure 0 0 0) 0x7fce080c8590 con 0x7fce18146830 2024-05-21T17:41:43.242 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:41:43.242 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/3741447020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3741447020},{"type":"v1","addr":"172.21.15.203:6827","nonce":3741447020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6810/205822275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":205822275},{"type":"v1","addr":"172.21.15.3:6810","nonce":205822275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-21T17:41:32.553094+0000","modified":"2024-05-21T17:41:35.579599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6811/1903526925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6809","nonce":1903526925},{"type":"v1","addr":"172.21.15.3:6811","nonce":1903526925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-05-21T17:41:43.242 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-21T17:41:43.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 10 mgrc shutdown 2024-05-21T17:41:43.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 1 -- 172.21.15.3:0/4221596474 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fcdfc0af260 msgr2=0x7fcdfc0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fcdfc0af260 0x7fcdfc0b1720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce14001c50 tx=0x7fce14009040 comp rx=0 tx=0).stop 2024-05-21T17:41:43.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 1 -- 172.21.15.3:0/4221596474 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce18146830 msgr2=0x7fce181dc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce18146830 0x7fce181dc1f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fce08037040 tx=0x7fce08004730 comp rx=0 tx=0).stop 2024-05-21T17:41:43.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 1 -- 172.21.15.3:0/4221596474 shutdown_connections 2024-05-21T17:41:43.244 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fcdfc0af260 0x7fcdfc0b1720 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce18133870 0x7fce18133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fce18146830 0x7fce181dc1f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 1 --2- 172.21.15.3:0/4221596474 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce04004800 0x7fce18135090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.242+0000 7fce1fb18640 1 -- 172.21.15.3:0/4221596474 >> 172.21.15.3:0/4221596474 conn(0x7fce180aab70 msgr2=0x7fce180bab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:43.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.243+0000 7fce1fb18640 1 -- 172.21.15.3:0/4221596474 shutdown_connections 2024-05-21T17:41:43.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.243+0000 7fce1fb18640 1 -- 172.21.15.3:0/4221596474 wait complete. 2024-05-21T17:41:43.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-21T17:41:32.553094+0000', 'modified': '2024-05-21T17:41:35.579599+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4245}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4245': {'gid': 4245, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.3:6811/1903526925', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.3:6809', 'nonce': 1903526925}, {'type': 'v1', 'addr': '172.21.15.3:6811', 'nonce': 1903526925}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4245, 'qdb_cluster': [4245]} 2024-05-21T17:41:43.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-21T17:41:43.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-21T17:41:43.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f0922fc7640 1 Processor -- start 2024-05-21T17:41:43.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f0922fc7640 1 -- start start 2024-05-21T17:41:43.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f0922fc7640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c146650 0x7f091c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f0922fc7640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f091c149e30 0x7f091c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f0922fc7640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f091c146b20 con 0x7f091c0ac330 2024-05-21T17:41:43.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f0922fc7640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f091c0a8530 con 0x7f091c149e30 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f0922fc7640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f091c146de0 con 0x7f091c146650 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f0920d63640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f091c0ac330 0x7f091c149720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:60782/0 (socket says 172.21.15.3:60782) 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f0920d63640 1 -- 172.21.15.3:0/1362472475 learned_addr learned my addr 172.21.15.3:0/1362472475 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f091bfff640 1 --2- 172.21.15.3:0/1362472475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c146650 0x7f091c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.663+0000 7f0921564640 1 --2- 172.21.15.3:0/1362472475 >> v2:172.21.15.3:3300/0 conn(0x7f091c149e30 0x7f091c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.664+0000 7f091b7fe640 1 -- 172.21.15.3:0/1362472475 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 530599406 0 0) 0x7f091c146b20 con 0x7f091c0ac330 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.664+0000 7f091bfff640 1 -- 172.21.15.3:0/1362472475 >> v2:172.21.15.3:3300/0 conn(0x7f091c149e30 msgr2=0x7f091c150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.664+0000 7f091bfff640 1 --2- 172.21.15.3:0/1362472475 >> v2:172.21.15.3:3300/0 conn(0x7f091c149e30 0x7f091c150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.664+0000 7f091bfff640 1 -- 172.21.15.3:0/1362472475 >> v1:172.21.15.3:6789/0 conn(0x7f091c0ac330 legacy=0x7f091c149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.664+0000 7f091bfff640 1 -- 172.21.15.3:0/1362472475 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c0ac710 con 0x7f091c146650 2024-05-21T17:41:43.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.664+0000 7f0921564640 1 --2- 172.21.15.3:0/1362472475 >> v2:172.21.15.3:3300/0 conn(0x7f091c149e30 0x7f091c150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.664+0000 7f091bfff640 1 --2- 172.21.15.3:0/1362472475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c146650 0x7f091c0ad480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f09100097a0 tx=0x7f091002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bfb9bd4d77b761d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f091b7fe640 1 -- 172.21.15.3:0/1362472475 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910031780 con 0x7f091c146650 2024-05-21T17:41:43.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f091b7fe640 1 -- 172.21.15.3:0/1362472475 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0910037070 con 0x7f091c146650 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f091b7fe640 1 -- 172.21.15.3:0/1362472475 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910031b40 con 0x7f091c146650 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f0922fc7640 1 -- 172.21.15.3:0/1362472475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c146650 msgr2=0x7f091c0ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f0922fc7640 1 --2- 172.21.15.3:0/1362472475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c146650 0x7f091c0ad480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f09100097a0 tx=0x7f091002b820 comp rx=0 tx=0).stop 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f0922fc7640 1 -- 172.21.15.3:0/1362472475 shutdown_connections 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f0922fc7640 1 --2- 172.21.15.3:0/1362472475 >> v2:172.21.15.3:3300/0 conn(0x7f091c149e30 0x7f091c150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f0922fc7640 1 --2- 172.21.15.3:0/1362472475 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c146650 0x7f091c0ad480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f0922fc7640 1 -- 172.21.15.3:0/1362472475 >> 172.21.15.3:0/1362472475 conn(0x7f091c0aab00 msgr2=0x7f091c0aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f0922fc7640 1 -- 172.21.15.3:0/1362472475 shutdown_connections 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.665+0000 7f0922fc7640 1 -- 172.21.15.3:0/1362472475 wait complete. 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.666+0000 7f0922fc7640 1 Processor -- start 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.666+0000 7f0922fc7640 1 -- start start 2024-05-21T17:41:43.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f0922fc7640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c0ac330 0x7f091c140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f0922fc7640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f091c146650 0x7f091c1dbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f0922fc7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f091c149e30 0x7f091c1d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f0922fc7640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f091c0aded0 con 0x7f091c146650 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f0922fc7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f091c06cf00 con 0x7f091c149e30 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f0922fc7640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f091c14a2c0 con 0x7f091c0ac330 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f091bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f091c146650 0x7f091c1dbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f0920d63640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c0ac330 0x7f091c140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f091bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f091c146650 0x7f091c1dbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51734/0 (socket says 172.21.15.3:51734) 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f0920d63640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c0ac330 0x7f091c140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41192/0 (socket says 172.21.15.3:41192) 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f091bfff640 1 -- 172.21.15.3:0/1452155821 learned_addr learned my addr 172.21.15.3:0/1452155821 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:43.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f0921564640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f091c149e30 0x7f091c1d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f091bfff640 1 -- 172.21.15.3:0/1452155821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c0ac330 msgr2=0x7f091c140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f091bfff640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c0ac330 0x7f091c140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f091bfff640 1 -- 172.21.15.3:0/1452155821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f091c149e30 msgr2=0x7f091c1d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f091bfff640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f091c149e30 0x7f091c1d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f091bfff640 1 -- 172.21.15.3:0/1452155821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c1dc220 con 0x7f091c146650 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.667+0000 7f0920d63640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c0ac330 0x7f091c140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.668+0000 7f091bfff640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f091c146650 0x7f091c1dbaf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0910034040 tx=0x7f09100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.668+0000 7f09197fa640 1 -- 172.21.15.3:0/1452155821 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910032610 con 0x7f091c146650 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.668+0000 7f09197fa640 1 -- 172.21.15.3:0/1452155821 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09100327b0 con 0x7f091c146650 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.668+0000 7f0922fc7640 1 -- 172.21.15.3:0/1452155821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091c1dc4e0 con 0x7f091c146650 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.668+0000 7f09197fa640 1 -- 172.21.15.3:0/1452155821 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910032ba0 con 0x7f091c146650 2024-05-21T17:41:43.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.668+0000 7f0922fc7640 1 -- 172.21.15.3:0/1452155821 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091c140170 con 0x7f091c146650 2024-05-21T17:41:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.670+0000 7f09197fa640 1 -- 172.21.15.3:0/1452155821 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f091004e440 con 0x7f091c146650 2024-05-21T17:41:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.670+0000 7f09197fa640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:41:43.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f0925bc4740 1 Processor -- start 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f0925bc4740 1 -- start start 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f09197fa640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f09197fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f09197fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f09197fa640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f09080af260 0x7f09080b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f09197fa640 1 -- 172.21.15.3:0/1452155821 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f09100fe5e0 con 0x7f091c146650 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5da7fd0 0x562ad5da73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc5530 0x562ad5dc59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc83d0 0x562ad5dc8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f0925bc4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562ad5d1bcf0 con 0x562ad5dc5530 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f0925bc4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562ad5d0b890 con 0x562ad5dc83d0 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f0925bc4740 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x562ad5d0f8d0 con 0x562ad5da7fd0 2024-05-21T17:41:43.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f0921564640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f09080af260 0x7f09080b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f091bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc5530 0x562ad5dc59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f0920d63640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5da7fd0 0x562ad5da73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f091bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc5530 0x562ad5dc59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51740/0 (socket says 172.21.15.3:51740) 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.671+0000 7f091bfff640 1 -- 172.21.15.3:0/3839023574 learned_addr learned my addr 172.21.15.3:0/3839023574 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f091bfff640 1 -- 172.21.15.3:0/3839023574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5da7fd0 msgr2=0x562ad5da73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f091bfff640 1 --2- 172.21.15.3:0/3839023574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5da7fd0 0x562ad5da73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f091bfff640 1 -- 172.21.15.3:0/3839023574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc83d0 msgr2=0x562ad5dc8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f0921564640 1 --2- 172.21.15.3:0/3839023574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc83d0 0x562ad5dc8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f091bfff640 1 --2- 172.21.15.3:0/3839023574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc83d0 0x562ad5dc8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f091bfff640 1 -- 172.21.15.3:0/3839023574 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ad5cd3eb0 con 0x562ad5dc5530 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f0920d63640 1 --2- 172.21.15.3:0/3839023574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5da7fd0 0x562ad5da73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:43.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f0921564640 1 --2- 172.21.15.3:0/3839023574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc83d0 0x562ad5dc8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f091bfff640 1 --2- 172.21.15.3:0/3839023574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc5530 0x562ad5dc59b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f09100fcfe0 tx=0x7f09100c76a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d9ef6378817f70 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f0921564640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f09080af260 0x7f09080b1720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f091c13dbf0 tx=0x7f0914009750 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f09027fc640 1 -- 172.21.15.3:0/3839023574 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100c8390 con 0x562ad5dc5530 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f09027fc640 1 -- 172.21.15.3:0/3839023574 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0910037070 con 0x562ad5dc5530 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f09027fc640 1 -- 172.21.15.3:0/3839023574 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100c7c90 con 0x562ad5dc5530 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f0925bc4740 1 -- 172.21.15.3:0/3839023574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc5530 msgr2=0x562ad5dc59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f0925bc4740 1 --2- 172.21.15.3:0/3839023574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc5530 0x562ad5dc59b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f09100fcfe0 tx=0x7f09100c76a0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f0925bc4740 1 -- 172.21.15.3:0/3839023574 shutdown_connections 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f0925bc4740 1 --2- 172.21.15.3:0/3839023574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc83d0 0x562ad5dc8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f0925bc4740 1 --2- 172.21.15.3:0/3839023574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc5530 0x562ad5dc59b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.672+0000 7f0925bc4740 1 --2- 172.21.15.3:0/3839023574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5da7fd0 0x562ad5da73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.673+0000 7f0925bc4740 1 -- 172.21.15.3:0/3839023574 >> 172.21.15.3:0/3839023574 conn(0x562ad5da5810 msgr2=0x562ad5da8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:43.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.673+0000 7f0925bc4740 1 -- 172.21.15.3:0/3839023574 shutdown_connections 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.673+0000 7f0925bc4740 1 -- 172.21.15.3:0/3839023574 wait complete. 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.673+0000 7f0925bc4740 1 -- start start 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.673+0000 7f0925bc4740 1 Processor -- start 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc5530 0x562ad5dd1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc83d0 0x562ad5dcf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dcfd60 0x562ad5dccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0925bc4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562ad59f1850 con 0x562ad5dc83d0 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0925bc4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562ad5caa4b0 con 0x562ad5dcfd60 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0925bc4740 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x562ad5d0b890 con 0x562ad5dc5530 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0925bc4740 10 client.0 _ll_register_callbacks cb 0x562ad5cda7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0920d63640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc5530 0x562ad5dd1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0920d63640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc5530 0x562ad5dd1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41200/0 (socket says 172.21.15.3:41200) 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0920d63640 1 -- 172.21.15.3:0/3128696493 learned_addr learned my addr 172.21.15.3:0/3128696493 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f091bfff640 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc83d0 0x562ad5dcf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0921564640 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dcfd60 0x562ad5dccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0920d63640 1 -- 172.21.15.3:0/3128696493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dcfd60 msgr2=0x562ad5dccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0920d63640 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dcfd60 0x562ad5dccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0920d63640 1 -- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc83d0 msgr2=0x562ad5dcf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0920d63640 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc83d0 0x562ad5dcf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0920d63640 1 -- 172.21.15.3:0/3128696493 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x562ad5a202d0 con 0x562ad5dc5530 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f0920d63640 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc5530 0x562ad5dd1250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f090c004e90 tx=0x7f090c001a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b709d66ef1959207 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.674+0000 7f08fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3301/0 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f08fb7fe640 1 -- 172.21.15.3:0/3128696493 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c012820 con 0x562ad5dc5530 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f08fb7fe640 1 -- 172.21.15.3:0/3128696493 <== mon.2 v2:172.21.15.3:3301/0 2 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f090c010040 con 0x562ad5dc5530 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f0925bc4740 1 -- 172.21.15.3:0/3128696493 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0910037070 con 0x562ad5dc5530 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f08fb7fe640 1 -- 172.21.15.3:0/3128696493 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f090c0129c0 con 0x562ad5dc5530 2024-05-21T17:41:43.685 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f0925bc4740 10 client.4388 fetch_fsmap learned FSMap version 8 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f0925bc4740 1 -- 172.21.15.3:0/3128696493 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x562ad5aea6b0 con 0x562ad5dc5530 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f08fb7fe640 1 -- 172.21.15.3:0/3128696493 <== mon.2 v2:172.21.15.3:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0910037070 con 0x562ad5dc5530 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f08fb7fe640 1 -- 172.21.15.3:0/3128696493 <== mon.2 v2:172.21.15.3:3301/0 5 ==== fsmap(e 8) ==== 2889+0+0 (secure 0 0 0) 0x7f090c01f400 con 0x562ad5dc5530 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f0925bc4740 10 client.4388 fetch_fsmap finished waiting for FSMap version 8 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f0925bc4740 10 client.4388 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f0925bc4740 20 client.4388 populate_metadata read hostname 'smithi003' 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f0925bc4740 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:6809/1903526925,v1:172.21.15.3:6811/1903526925] conn(0x562ad5df2500 0x562ad5df48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f0925bc4740 4 client.4388 mds_command: new command op to 4245 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f0925bc4740 1 -- 172.21.15.3:0/3128696493 --> [v2:172.21.15.3:6809/1903526925,v1:172.21.15.3:6811/1903526925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ad5d1dc40 con 0x562ad5df2500 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.675+0000 7f091bfff640 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:6809/1903526925,v1:172.21.15.3:6811/1903526925] conn(0x562ad5df2500 0x562ad5df48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f08fb7fe640 10 client.4388 ms_handle_connect on v2:172.21.15.3:6809/1903526925 2024-05-21T17:41:43.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f091bfff640 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:6809/1903526925,v1:172.21.15.3:6811/1903526925] conn(0x562ad5df2500 0x562ad5df48f0 crc :-1 s=READY pgs=9 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b2480a0e6850db server_cookie=5c976d84c0d21d05 in_seq=0 out_seq=0 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f08fb7fe640 1 -- 172.21.15.3:0/3128696493 <== mds.0 v2:172.21.15.3:6809/1903526925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562ad5d1dc40 con 0x562ad5df2500 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f08fb7fe640 10 client.4388 handle_command_reply: tid=0 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f0925bc4740 1 client.4388 shutdown 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f0925bc4740 1 -- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc5530 msgr2=0x562ad5dd1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f0925bc4740 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc5530 0x562ad5dd1250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f090c004e90 tx=0x7f090c001a70 comp rx=0 tx=0).stop 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f0925bc4740 1 -- 172.21.15.3:0/3128696493 shutdown_connections 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f0925bc4740 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:6809/1903526925,v1:172.21.15.3:6811/1903526925] conn(0x562ad5df2500 0x562ad5df48f0 crc :-1 s=READY pgs=9 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f0925bc4740 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dcfd60 0x562ad5dccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f0925bc4740 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dc83d0 0x562ad5dcf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f0925bc4740 1 --2- 172.21.15.3:0/3128696493 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc5530 0x562ad5dd1250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f0925bc4740 1 -- 172.21.15.3:0/3128696493 >> 172.21.15.3:0/3128696493 conn(0x562ad5da7fd0 msgr2=0x562ad5dcbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f08fb7fe640 0 client.4388 ms_handle_reset on v2:172.21.15.3:6809/1903526925 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.676+0000 7f0925bc4740 1 -- 172.21.15.3:0/3128696493 shutdown_connections 2024-05-21T17:41:43.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.677+0000 7f0925bc4740 1 -- 172.21.15.3:0/3128696493 wait complete. 2024-05-21T17:41:43.688 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.677+0000 7f0925bc4740 20 client.4388 trim_cache size 0 max 16384 2024-05-21T17:41:43.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.695+0000 7f0925bc4740 1 Processor -- start 2024-05-21T17:41:43.696 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.695+0000 7f0925bc4740 1 -- start start 2024-05-21T17:41:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc5530 0x562ad5da83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.698 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dcb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5da8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0925bc4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562ad5dd9500 con 0x562ad5dcfd60 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0925bc4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1e8820 con 0x562ad5dc5530 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0925bc4740 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f091c1493e0 con 0x562ad5dc83d0 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f091bfff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dcb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0921564640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5da8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0921564640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5da8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51770/0 (socket says 172.21.15.3:51770) 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0921564640 1 -- 172.21.15.3:0/1189814108 learned_addr learned my addr 172.21.15.3:0/1189814108 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f091bfff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dcb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41216/0 (socket says 172.21.15.3:41216) 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f091bfff640 1 -- 172.21.15.3:0/1189814108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc5530 msgr2=0x562ad5da83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f091bfff640 1 --2- 172.21.15.3:0/1189814108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc5530 0x562ad5da83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f091bfff640 1 -- 172.21.15.3:0/1189814108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 msgr2=0x562ad5da8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f091bfff640 1 --2- 172.21.15.3:0/1189814108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5da8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f091bfff640 1 -- 172.21.15.3:0/1189814108 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ad5d9f890 con 0x562ad5dc83d0 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0921564640 1 --2- 172.21.15.3:0/1189814108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5da8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f091bfff640 1 --2- 172.21.15.3:0/1189814108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dcb0e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f09100fd070 tx=0x7f09100e9000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6a388f75a5eb484 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0901ffb640 1 -- 172.21.15.3:0/1189814108 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100cc040 con 0x562ad5dc83d0 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0901ffb640 1 -- 172.21.15.3:0/1189814108 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09100e5030 con 0x562ad5dc83d0 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.696+0000 7f0901ffb640 1 -- 172.21.15.3:0/1189814108 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100d0af0 con 0x562ad5dc83d0 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.697+0000 7f0925bc4740 1 -- 172.21.15.3:0/1189814108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 msgr2=0x562ad5dcb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.697+0000 7f0925bc4740 1 --2- 172.21.15.3:0/1189814108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dcb0e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f09100fd070 tx=0x7f09100e9000 comp rx=0 tx=0).stop 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.697+0000 7f0925bc4740 1 -- 172.21.15.3:0/1189814108 shutdown_connections 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.697+0000 7f0925bc4740 1 --2- 172.21.15.3:0/1189814108 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5da8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.697+0000 7f0925bc4740 1 --2- 172.21.15.3:0/1189814108 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dcb0e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.697+0000 7f0925bc4740 1 --2- 172.21.15.3:0/1189814108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc5530 0x562ad5da83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.697+0000 7f0925bc4740 1 -- 172.21.15.3:0/1189814108 >> 172.21.15.3:0/1189814108 conn(0x562ad5da7fd0 msgr2=0x562ad5dcbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.697+0000 7f0925bc4740 1 -- 172.21.15.3:0/1189814108 shutdown_connections 2024-05-21T17:41:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.697+0000 7f0925bc4740 1 -- 172.21.15.3:0/1189814108 wait complete. 2024-05-21T17:41:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.697+0000 7f0925bc4740 1 -- start start 2024-05-21T17:41:43.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f0925bc4740 1 Processor -- start 2024-05-21T17:41:43.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc5530 0x562ad5da9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f0925bc4740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5ddb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f0925bc4740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562ad5d1bcf0 con 0x562ad5dcfd60 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f0925bc4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1e7930 con 0x562ad5dc5530 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f0925bc4740 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x562ad5dd17e0 con 0x562ad5dc83d0 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f0925bc4740 10 client.0 _ll_register_callbacks cb 0x7f090c007290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f091bfff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f0921564640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5ddb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f091bfff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dec9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:41226/0 (socket says 172.21.15.3:41226) 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f0921564640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5ddb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51784/0 (socket says 172.21.15.3:51784) 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.698+0000 7f091bfff640 1 -- 172.21.15.3:0/4018978797 learned_addr learned my addr 172.21.15.3:0/4018978797 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f0920d63640 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc5530 0x562ad5da9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f091bfff640 1 -- 172.21.15.3:0/4018978797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc5530 msgr2=0x562ad5da9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f091bfff640 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc5530 0x562ad5da9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f091bfff640 1 -- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 msgr2=0x562ad5ddb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f091bfff640 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5ddb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f091bfff640 1 -- 172.21.15.3:0/4018978797 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x562ad5dec110 con 0x562ad5dc83d0 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f091bfff640 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dec9a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f091002be30 tx=0x7f091002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1b9c9a1d1e5a6db server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f08fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.3:3301/0 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f08fb7fe640 1 -- 172.21.15.3:0/4018978797 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100c7c90 con 0x562ad5dc83d0 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f08fb7fe640 1 -- 172.21.15.3:0/4018978797 <== mon.2 v2:172.21.15.3:3301/0 2 ==== osd_map(16..16 src has 1..16) ==== 4334+0+0 (secure 0 0 0) 0x7f09100c7ef0 con 0x562ad5dc83d0 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f0925bc4740 1 -- 172.21.15.3:0/4018978797 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09100e5030 con 0x562ad5dc83d0 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f08fb7fe640 1 -- 172.21.15.3:0/4018978797 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f09100d0af0 con 0x562ad5dc83d0 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f0925bc4740 10 client.4394 fetch_fsmap learned FSMap version 8 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f0925bc4740 1 -- 172.21.15.3:0/4018978797 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x562ad5e49660 con 0x562ad5dc83d0 2024-05-21T17:41:43.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f08fb7fe640 1 -- 172.21.15.3:0/4018978797 <== mon.2 v2:172.21.15.3:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09100e5030 con 0x562ad5dc83d0 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f08fb7fe640 1 -- 172.21.15.3:0/4018978797 <== mon.2 v2:172.21.15.3:3301/0 5 ==== fsmap(e 8) ==== 2889+0+0 (secure 0 0 0) 0x7f09100d0660 con 0x562ad5dc83d0 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f0925bc4740 10 client.4394 fetch_fsmap finished waiting for FSMap version 8 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f0925bc4740 10 client.4394 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f0925bc4740 20 client.4394 populate_metadata read hostname 'smithi003' 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f0925bc4740 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:6809/1903526925,v1:172.21.15.3:6811/1903526925] conn(0x562ad5dce3c0 0x562ad5dce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f0925bc4740 4 client.4394 mds_command: new command op to 4245 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.699+0000 7f0925bc4740 1 -- 172.21.15.3:0/4018978797 --> [v2:172.21.15.3:6809/1903526925,v1:172.21.15.3:6811/1903526925] -- command(tid 0: {"prefix": "status"}) -- 0x562ad5da5fd0 con 0x562ad5dce3c0 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0920d63640 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:6809/1903526925,v1:172.21.15.3:6811/1903526925] conn(0x562ad5dce3c0 0x562ad5dce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f08fb7fe640 10 client.4394 ms_handle_connect on v2:172.21.15.3:6809/1903526925 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0920d63640 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:6809/1903526925,v1:172.21.15.3:6811/1903526925] conn(0x562ad5dce3c0 0x562ad5dce7a0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ef138b161d8c7f server_cookie=2b69fb1ac821d4fd in_seq=0 out_seq=0 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f08fb7fe640 1 -- 172.21.15.3:0/4018978797 <== mds.0 v2:172.21.15.3:6809/1903526925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x562ad5da5fd0 con 0x562ad5dce3c0 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f08fb7fe640 10 client.4394 handle_command_reply: tid=0 2024-05-21T17:41:43.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 client.4394 shutdown 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 -- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 msgr2=0x562ad5dec9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dec9a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f091002be30 tx=0x7f091002b820 comp rx=0 tx=0).stop 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 -- 172.21.15.3:0/4018978797 shutdown_connections 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562ad5dcfd60 0x562ad5ddb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x562ad5dc83d0 0x562ad5dec9a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.3:6809/1903526925,v1:172.21.15.3:6811/1903526925] conn(0x562ad5dce3c0 0x562ad5dce7a0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 --2- 172.21.15.3:0/4018978797 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ad5dc5530 0x562ad5da9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f08fb7fe640 0 client.4394 ms_handle_reset on v2:172.21.15.3:6809/1903526925 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 -- 172.21.15.3:0/4018978797 >> 172.21.15.3:0/4018978797 conn(0x562ad5da7fd0 msgr2=0x562ad5da83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 -- 172.21.15.3:0/4018978797 shutdown_connections 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 1 -- 172.21.15.3:0/4018978797 wait complete. 2024-05-21T17:41:43.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.700+0000 7f0925bc4740 20 client.4394 trim_cache size 0 max 16384 2024-05-21T17:41:43.709 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-21T17:41:43.709 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442", 2024-05-21T17:41:43.709 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-05-21T17:41:43.709 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 4245, 2024-05-21T17:41:43.709 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-05-21T17:41:43.709 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-05-21T17:41:43.709 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-05-21T17:41:43.709 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 10.525887074, 2024-05-21T17:41:43.709 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 8, 2024-05-21T17:41:43.709 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 16, 2024-05-21T17:41:43.710 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 16, 2024-05-21T17:41:43.710 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 16.447552871999999 2024-05-21T17:41:43.710 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-21T17:41:43.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 10 mgrc shutdown 2024-05-21T17:41:43.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 -- 172.21.15.3:0/1452155821 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f09080af260 msgr2=0x7f09080b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f09080af260 0x7f09080b1720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f091c13dbf0 tx=0x7f0914009750 comp rx=0 tx=0).stop 2024-05-21T17:41:43.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 -- 172.21.15.3:0/1452155821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f091c146650 msgr2=0x7f091c1dbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:41:43.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f091c146650 0x7f091c1dbaf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0910034040 tx=0x7f09100046c0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 -- 172.21.15.3:0/1452155821 shutdown_connections 2024-05-21T17:41:43.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f09080af260 0x7f09080b1720 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f091c149e30 0x7f091c1d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f091c146650 0x7f091c1dbaf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 --2- 172.21.15.3:0/1452155821 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f091c0ac330 0x7f091c140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:41:43.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 -- 172.21.15.3:0/1452155821 >> 172.21.15.3:0/1452155821 conn(0x7f091c0aab00 msgr2=0x7f091c0bc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:41:43.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 -- 172.21.15.3:0/1452155821 shutdown_connections 2024-05-21T17:41:43.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:41:43.701+0000 7f0922fc7640 1 -- 172.21.15.3:0/1452155821 wait complete. 2024-05-21T17:41:43.713 INFO:teuthology.run_tasks:Running task sequential... 2024-05-21T17:41:43.722 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-21T17:41:43.722 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-21T17:41:43.722 INFO:teuthology.task.print:Enabling mgr modules 2024-05-21T17:41:43.723 INFO:teuthology.run_tasks:Running task workunit... 2024-05-21T17:41:43.744 INFO:tasks.workunit:Pulling workunits from ref 9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 2024-05-21T17:41:43.745 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-21T17:41:43.745 DEBUG:teuthology.orchestra.run.smithi003:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-21T17:41:43.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T17:41:43.762 INFO:teuthology.orchestra.run.smithi003.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-21T17:41:43.762 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-21T17:41:43.819 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-21T17:41:43.819 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-21T17:41:43.874 INFO:tasks.workunit:timeout=3h 2024-05-21T17:41:43.877 INFO:tasks.workunit:cleanup=True 2024-05-21T17:41:43.877 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 2024-05-21T17:41:43.931 INFO:tasks.workunit.client.0.smithi003.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-21T17:41:56.869 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-21T17:41:56.872 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-21T17:42:26.958 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-21T17:42:26.962 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-21T17:42:32.175 INFO:tasks.workunit.client.0.smithi003.stderr:Updating files: 96% (11668/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-05-21T17:42:32.734 INFO:tasks.workunit.client.0.smithi003.stderr:Note: switching to '9ef9bcfdb32ec4999d7ca025815290e8395c6ae7'. 2024-05-21T17:42:32.734 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-21T17:42:32.734 INFO:tasks.workunit.client.0.smithi003.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-21T17:42:32.734 INFO:tasks.workunit.client.0.smithi003.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-21T17:42:32.734 INFO:tasks.workunit.client.0.smithi003.stderr:state without impacting any branches by switching back to a branch. 2024-05-21T17:42:32.734 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-21T17:42:32.734 INFO:tasks.workunit.client.0.smithi003.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr: git switch -c 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr:Or undo this operation with: 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr: git switch - 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-21T17:42:32.735 INFO:tasks.workunit.client.0.smithi003.stderr:HEAD is now at 9ef9bcfdb32 Merge branch 'asan_shec' of https://github.com/Svelar/ceph into wip-yuri7-testing-2024-05-20-1227 2024-05-21T17:42:32.740 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-21T17:42:32.771 INFO:tasks.workunit.client.0.smithi003.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-21T17:42:32.773 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-21T17:42:32.773 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-21T17:42:33.362 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-21T17:42:33.422 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-21T17:42:33.464 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-21T17:42:33.465 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-21T17:42:33.465 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-21T17:42:33.506 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-21T17:42:33.510 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-21T17:42:33.510 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-21T17:42:33.566 INFO:tasks.workunit:Running workunits matching mgr/test_localpool.sh on client.0... 2024-05-21T17:42:33.568 INFO:tasks.workunit:Running workunit mgr/test_localpool.sh... 2024-05-21T17:42:33.568 DEBUG:teuthology.orchestra.run.smithi003:workunit test mgr/test_localpool.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ef9bcfdb32ec4999d7ca025815290e8395c6ae7 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mgr/test_localpool.sh 2024-05-21T17:42:33.627 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph config set mgr mgr/localpool/subtree host 2024-05-21T17:42:33.712 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.709+0000 7fdfafc45640 1 Processor -- start 2024-05-21T17:42:33.712 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfafc45640 1 -- start start 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfafc45640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdfa8075690 0x7fdfa80a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfafc45640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 0x7fdfa8145a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfafc45640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa8146550 con 0x7fdfa8145fa0 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfafc45640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa8075a70 con 0x7fdfa8075690 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfafc45640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa80a5060 con 0x7fdfa80a90f0 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfad1e0640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 0x7fdfa8145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfad9e1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdfa8075690 0x7fdfa80a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfad9e1640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fdfa8075690 0x7fdfa80a9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57830/0 (socket says 172.21.15.3:57830) 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfad1e0640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 0x7fdfa8145a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40810/0 (socket says 172.21.15.3:40810) 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfad9e1640 1 -- 172.21.15.3:0/2726412406 learned_addr learned my addr 172.21.15.3:0/2726412406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:33.714 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.710+0000 7fdfae1e2640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fdfa8145fa0 0x7fdfa806e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:54616/0 (socket says 172.21.15.3:54616) 2024-05-21T17:42:33.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.711+0000 7fdfad9e1640 1 -- 172.21.15.3:0/2726412406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 msgr2=0x7fdfa8145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:33.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.711+0000 7fdfad9e1640 1 --2- 172.21.15.3:0/2726412406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 0x7fdfa8145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:33.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.711+0000 7fdfad9e1640 1 -- 172.21.15.3:0/2726412406 >> v1:172.21.15.3:6789/0 conn(0x7fdfa8145fa0 legacy=0x7fdfa806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:33.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.711+0000 7fdfad9e1640 1 -- 172.21.15.3:0/2726412406 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa8146810 con 0x7fdfa8075690 2024-05-21T17:42:33.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.711+0000 7fdfac9df640 1 -- 172.21.15.3:0/2726412406 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569941216 0 0) 0x7fdfa8146550 con 0x7fdfa8145fa0 2024-05-21T17:42:33.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.711+0000 7fdfad9e1640 1 --2- 172.21.15.3:0/2726412406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 0x7fdfa80a9d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009870 tx=0x7fdfa402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab78ebd3f1a4645c server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:33.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfac9df640 1 -- 172.21.15.3:0/2726412406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4035020 con 0x7fdfa8075690 2024-05-21T17:42:33.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfac9df640 1 -- 172.21.15.3:0/2726412406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa4031640 con 0x7fdfa8075690 2024-05-21T17:42:33.715 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfac9df640 1 -- 172.21.15.3:0/2726412406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa402d040 con 0x7fdfa8075690 2024-05-21T17:42:33.718 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfafc45640 1 -- 172.21.15.3:0/2726412406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 msgr2=0x7fdfa80a9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:33.718 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfafc45640 1 --2- 172.21.15.3:0/2726412406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 0x7fdfa80a9d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4009870 tx=0x7fdfa402b8f0 comp rx=0 tx=0).stop 2024-05-21T17:42:33.718 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfafc45640 1 -- 172.21.15.3:0/2726412406 shutdown_connections 2024-05-21T17:42:33.718 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfafc45640 1 --2- 172.21.15.3:0/2726412406 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 0x7fdfa8145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:33.718 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfafc45640 1 --2- 172.21.15.3:0/2726412406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 0x7fdfa80a9d40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:33.718 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfafc45640 1 -- 172.21.15.3:0/2726412406 >> 172.21.15.3:0/2726412406 conn(0x7fdfa80aa9e0 msgr2=0x7fdfa80a7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:33.718 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfafc45640 1 -- 172.21.15.3:0/2726412406 shutdown_connections 2024-05-21T17:42:33.718 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.712+0000 7fdfafc45640 1 -- 172.21.15.3:0/2726412406 wait complete. 2024-05-21T17:42:33.718 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.713+0000 7fdfafc45640 1 Processor -- start 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.713+0000 7fdfafc45640 1 -- start start 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfafc45640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 0x7fdfa8073970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfafc45640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 0x7fdfa80a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfafc45640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfa8145fa0 0x7fdfa80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfafc45640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa80aa280 con 0x7fdfa8075690 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfafc45640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa80a6b50 con 0x7fdfa8145fa0 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfafc45640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa80aa400 con 0x7fdfa80a90f0 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfad9e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 0x7fdfa8073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfad1e0640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 0x7fdfa80a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfad9e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 0x7fdfa8073970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57832/0 (socket says 172.21.15.3:57832) 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfad9e1640 1 -- 172.21.15.3:0/2015486021 learned_addr learned my addr 172.21.15.3:0/2015486021 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:33.719 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfad9e1640 1 -- 172.21.15.3:0/2015486021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 msgr2=0x7fdfa80a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfad9e1640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 0x7fdfa80a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfae1e2640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfa8145fa0 0x7fdfa80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfad9e1640 1 -- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfa8145fa0 msgr2=0x7fdfa80a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfad9e1640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfa8145fa0 0x7fdfa80a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfad9e1640 1 -- 172.21.15.3:0/2015486021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa80a2330 con 0x7fdfa8075690 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfad1e0640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 0x7fdfa80a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.714+0000 7fdfae1e2640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfa8145fa0 0x7fdfa80a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.715+0000 7fdfad9e1640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 0x7fdfa8073970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdfa402b930 tx=0x7fdfa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.715+0000 7fdf9e7fc640 1 -- 172.21.15.3:0/2015486021 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4040070 con 0x7fdfa8075690 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.715+0000 7fdf9e7fc640 1 -- 172.21.15.3:0/2015486021 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa403e070 con 0x7fdfa8075690 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.715+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa80a25f0 con 0x7fdfa8075690 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.715+0000 7fdf9e7fc640 1 -- 172.21.15.3:0/2015486021 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4035040 con 0x7fdfa8075690 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.715+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa80a44c0 con 0x7fdfa8075690 2024-05-21T17:42:33.720 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.716+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa8069f80 con 0x7fdfa8075690 2024-05-21T17:42:33.721 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.717+0000 7fdf9e7fc640 1 -- 172.21.15.3:0/2015486021 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7fdfa4038020 con 0x7fdfa8075690 2024-05-21T17:42:33.721 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.717+0000 7fdf9e7fc640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:42:33.722 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.718+0000 7fdf9e7fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:42:33.722 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.718+0000 7fdf9e7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:33.722 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.718+0000 7fdf9e7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:33.722 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.718+0000 7fdf9e7fc640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fdf8c0af260 0x7fdf8c0b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:33.722 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.718+0000 7fdf9e7fc640 1 -- 172.21.15.3:0/2015486021 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4450+0+0 (secure 0 0 0) 0x7fdfa40fe5b0 con 0x7fdfa8075690 2024-05-21T17:42:33.722 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.718+0000 7fdfad1e0640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fdf8c0af260 0x7fdf8c0b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:33.722 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.719+0000 7fdfad1e0640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fdf8c0af260 0x7fdf8c0b1720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8149d40 tx=0x7fdf98009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:33.722 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.721+0000 7fdf9e7fc640 1 -- 172.21.15.3:0/2015486021 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdfa40c82d0 con 0x7fdfa8075690 2024-05-21T17:42:33.914 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.912+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/localpool/subtree}] v 0) -- 0x7fdfa80a51f0 con 0x7fdfa8075690 2024-05-21T17:42:33.919 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:33.917+0000 7fdf9e7fc640 1 -- 172.21.15.3:0/2015486021 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/localpool/subtree}]=0 v6)=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7fdfa40d02c0 con 0x7fdfa8075690 2024-05-21T17:42:34.033 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.031+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/localpool/subtree}] v 0) -- 0x7fdfa80762a0 con 0x7fdfa8075690 2024-05-21T17:42:34.034 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.031+0000 7fdf9e7fc640 1 -- 172.21.15.3:0/2015486021 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/localpool/subtree}]=0 v6)=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7fdfa40c7bd0 con 0x7fdfa8075690 2024-05-21T17:42:34.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 10 mgrc shutdown 2024-05-21T17:42:34.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fdf8c0af260 msgr2=0x7fdf8c0b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fdf8c0af260 0x7fdf8c0b1720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8149d40 tx=0x7fdf98009190 comp rx=0 tx=0).stop 2024-05-21T17:42:34.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 msgr2=0x7fdfa8073970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.036 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 0x7fdfa8073970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdfa402b930 tx=0x7fdfa40046c0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 shutdown_connections 2024-05-21T17:42:34.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7fdf8c0af260 0x7fdf8c0b1720 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfa8145fa0 0x7fdfa80a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fdfa80a90f0 0x7fdfa80a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 --2- 172.21.15.3:0/2015486021 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdfa8075690 0x7fdfa8073970 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 >> 172.21.15.3:0/2015486021 conn(0x7fdfa80aa9e0 msgr2=0x7fdfa80a7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:34.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 shutdown_connections 2024-05-21T17:42:34.037 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.034+0000 7fdfafc45640 1 -- 172.21.15.3:0/2015486021 wait complete. 2024-05-21T17:42:34.045 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph config set mgr mgr/localpool/failure_domain osd 2024-05-21T17:42:34.129 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.127+0000 7f39098aa640 1 Processor -- start 2024-05-21T17:42:34.129 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.127+0000 7f39098aa640 1 -- start start 2024-05-21T17:42:34.131 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39098aa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f39040a7030 0x7f39040a8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.131 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39098aa640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39040a6880 0x7f39041435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.131 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39098aa640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3904143b00 0x7f3904072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.131 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39098aa640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f3904070190 con 0x7f3904143b00 2024-05-21T17:42:34.131 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39098aa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f39040b9860 con 0x7f39040a7030 2024-05-21T17:42:34.131 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39098aa640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f39040a9fd0 con 0x7f39040a6880 2024-05-21T17:42:34.131 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39037fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3904143b00 0x7f3904072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.131 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39027fc640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39040a6880 0x7f39041435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.131 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39037fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f3904143b00 0x7f3904072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57836/0 (socket says 172.21.15.3:57836) 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39027fc640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39040a6880 0x7f39041435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40830/0 (socket says 172.21.15.3:40830) 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39037fe640 1 -- 172.21.15.3:0/2762745558 learned_addr learned my addr 172.21.15.3:0/2762745558 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f3902ffd640 1 --2- 172.21.15.3:0/2762745558 >> v2:172.21.15.203:3300/0 conn(0x7f39040a7030 0x7f39040a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39037fe640 1 -- 172.21.15.3:0/2762745558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39040a6880 msgr2=0x7f39041435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39037fe640 1 --2- 172.21.15.3:0/2762745558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39040a6880 0x7f39041435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39037fe640 1 -- 172.21.15.3:0/2762745558 >> v2:172.21.15.203:3300/0 conn(0x7f39040a7030 msgr2=0x7f39040a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39037fe640 1 --2- 172.21.15.3:0/2762745558 >> v2:172.21.15.203:3300/0 conn(0x7f39040a7030 0x7f39040a8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39037fe640 1 -- 172.21.15.3:0/2762745558 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39040b9a90 con 0x7f3904143b00 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.128+0000 7f39027fc640 1 --2- 172.21.15.3:0/2762745558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39040a6880 0x7f39041435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.129+0000 7f39037fe640 1 --2- 172.21.15.3:0/2762745558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3904143b00 0x7f3904072de0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3904072a10 tx=0x7f38fc02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15d84657fcae88d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.129+0000 7f3901ffb640 1 -- 172.21.15.3:0/2762745558 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc02c950 con 0x7f3904143b00 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.129+0000 7f3901ffb640 1 -- 172.21.15.3:0/2762745558 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc02caf0 con 0x7f3904143b00 2024-05-21T17:42:34.132 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.129+0000 7f3901ffb640 1 -- 172.21.15.3:0/2762745558 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0343f0 con 0x7f3904143b00 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.129+0000 7f39098aa640 1 -- 172.21.15.3:0/2762745558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3904143b00 msgr2=0x7f3904072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.129+0000 7f39098aa640 1 --2- 172.21.15.3:0/2762745558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3904143b00 0x7f3904072de0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3904072a10 tx=0x7f38fc02bdb0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.129+0000 7f39098aa640 1 -- 172.21.15.3:0/2762745558 shutdown_connections 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.129+0000 7f39098aa640 1 --2- 172.21.15.3:0/2762745558 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3904143b00 0x7f3904072de0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.129+0000 7f39098aa640 1 --2- 172.21.15.3:0/2762745558 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f39040a6880 0x7f39041435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.129+0000 7f39098aa640 1 --2- 172.21.15.3:0/2762745558 >> v2:172.21.15.203:3300/0 conn(0x7f39040a7030 0x7f39040a8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.130+0000 7f39098aa640 1 -- 172.21.15.3:0/2762745558 >> 172.21.15.3:0/2762745558 conn(0x7f39040724c0 msgr2=0x7f3904075420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.130+0000 7f39098aa640 1 -- 172.21.15.3:0/2762745558 shutdown_connections 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.130+0000 7f39098aa640 1 -- 172.21.15.3:0/2762745558 wait complete. 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f39098aa640 1 Processor -- start 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f39098aa640 1 -- start start 2024-05-21T17:42:34.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f39098aa640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39040a6880 0x7f390413ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f39098aa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39040a7030 0x7f3904148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f39098aa640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3904143b00 0x7f39040b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f39098aa640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3904143f90 con 0x7f39040a6880 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f39098aa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f39040b9d80 con 0x7f39040a7030 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f39098aa640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f39040a7410 con 0x7f3904143b00 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f3902ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39040a6880 0x7f390413ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f39037fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3904143b00 0x7f39040b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f3902ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39040a6880 0x7f390413ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57840/0 (socket says 172.21.15.3:57840) 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.131+0000 7f3902ffd640 1 -- 172.21.15.3:0/1969269022 learned_addr learned my addr 172.21.15.3:0/1969269022 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f39027fc640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39040a7030 0x7f3904148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f3902ffd640 1 -- 172.21.15.3:0/1969269022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3904143b00 msgr2=0x7f39040b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f3902ffd640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3904143b00 0x7f39040b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f3902ffd640 1 -- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39040a7030 msgr2=0x7f3904148120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f3902ffd640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39040a7030 0x7f3904148120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f3902ffd640 1 -- 172.21.15.3:0/1969269022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39041469a0 con 0x7f39040a6880 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f39037fe640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3904143b00 0x7f39040b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f39027fc640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39040a7030 0x7f3904148120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f3902ffd640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39040a6880 0x7f390413ef50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f38f400d830 tx=0x7f38f400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f38ebfff640 1 -- 172.21.15.3:0/1969269022 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f400b6c0 con 0x7f39040a6880 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f38ebfff640 1 -- 172.21.15.3:0/1969269022 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f4019070 con 0x7f39040a6880 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3904146c60 con 0x7f39040a6880 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f38ebfff640 1 -- 172.21.15.3:0/1969269022 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f400ba80 con 0x7f39040a6880 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.132+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390406f2b0 con 0x7f39040a6880 2024-05-21T17:42:34.138 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.133+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39040507e0 con 0x7f39040a6880 2024-05-21T17:42:34.139 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.134+0000 7f38ebfff640 1 -- 172.21.15.3:0/1969269022 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f38f4028430 con 0x7f39040a6880 2024-05-21T17:42:34.139 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.134+0000 7f38ebfff640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:42:34.139 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.135+0000 7f38ebfff640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:42:34.139 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.135+0000 7f38ebfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:34.139 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.135+0000 7f38ebfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:34.140 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.135+0000 7f38ebfff640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f38f00af260 0x7f38f00b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.140 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.135+0000 7f38ebfff640 1 -- 172.21.15.3:0/1969269022 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4450+0+0 (secure 0 0 0) 0x7f38f40ddb20 con 0x7f39040a6880 2024-05-21T17:42:34.140 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.135+0000 7f39027fc640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f38f00af260 0x7f38f00b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.140 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.136+0000 7f39027fc640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f38f00af260 0x7f38f00b1720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39040a8da0 tx=0x7f38f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:34.140 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.138+0000 7f38ebfff640 1 -- 172.21.15.3:0/1969269022 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f38f40a77b0 con 0x7f39040a6880 2024-05-21T17:42:34.330 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.328+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/localpool/failure_domain}] v 0) -- 0x7f39040ab570 con 0x7f39040a6880 2024-05-21T17:42:34.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.333+0000 7f38ebfff640 1 -- 172.21.15.3:0/1969269022 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/localpool/failure_domain}]=0 v7)=0 v7) ==== 128+0+0 (secure 0 0 0) 0x7f38f40a70b0 con 0x7f39040a6880 2024-05-21T17:42:34.447 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.444+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/localpool/failure_domain}] v 0) -- 0x7f390406fbd0 con 0x7f39040a6880 2024-05-21T17:42:34.447 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.445+0000 7f38ebfff640 1 -- 172.21.15.3:0/1969269022 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/localpool/failure_domain}]=0 v7)=0 v7) ==== 128+0+0 (secure 0 0 0) 0x7f38f40ab460 con 0x7f39040a6880 2024-05-21T17:42:34.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.446+0000 7f39098aa640 10 mgrc shutdown 2024-05-21T17:42:34.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f38f00af260 msgr2=0x7f38f00b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f38f00af260 0x7f38f00b1720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39040a8da0 tx=0x7f38f8008040 comp rx=0 tx=0).stop 2024-05-21T17:42:34.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39040a6880 msgr2=0x7f390413ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39040a6880 0x7f390413ef50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f38f400d830 tx=0x7f38f400dcf0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 shutdown_connections 2024-05-21T17:42:34.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f38f00af260 0x7f38f00b1720 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.450 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3904143b00 0x7f39040b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.450 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39040a7030 0x7f3904148120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.450 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 --2- 172.21.15.3:0/1969269022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39040a6880 0x7f390413ef50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.450 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 >> 172.21.15.3:0/1969269022 conn(0x7f39040724c0 msgr2=0x7f39040a83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:34.450 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 shutdown_connections 2024-05-21T17:42:34.450 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.447+0000 7f39098aa640 1 -- 172.21.15.3:0/1969269022 wait complete. 2024-05-21T17:42:34.458 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph mgr module enable localpool 2024-05-21T17:42:34.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.539+0000 7f816248e640 1 Processor -- start 2024-05-21T17:42:34.541 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.539+0000 7f816248e640 1 -- start start 2024-05-21T17:42:34.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f816248e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f815c0ac660 0x7f815c06c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f816248e640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f815c0aa800 0x7f815c146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f816248e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f815c146670 con 0x7f815c146fe0 2024-05-21T17:42:34.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f816248e640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f815c0ac210 con 0x7f815c0aa800 2024-05-21T17:42:34.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f816248e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f815c146360 con 0x7f815c0ac660 2024-05-21T17:42:34.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f815b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f815c0aa800 0x7f815c146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f815b7fe640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f815c0aa800 0x7f815c146aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57844/0 (socket says 172.21.15.3:57844) 2024-05-21T17:42:34.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f815b7fe640 1 -- 172.21.15.3:0/3428457669 learned_addr learned my addr 172.21.15.3:0/3428457669 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:34.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f815bfff640 1 --2- 172.21.15.3:0/3428457669 >> v2:172.21.15.203:3300/0 conn(0x7f815c0ac660 0x7f815c06c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.543 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f815b7fe640 1 -- 172.21.15.3:0/3428457669 >> v2:172.21.15.203:3300/0 conn(0x7f815c0ac660 msgr2=0x7f815c06c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:34.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f815b7fe640 1 --2- 172.21.15.3:0/3428457669 >> v2:172.21.15.203:3300/0 conn(0x7f815c0ac660 0x7f815c06c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f815b7fe640 1 -- 172.21.15.3:0/3428457669 >> v1:172.21.15.203:6789/0 conn(0x7f815c146fe0 legacy=0x7f815c0ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.540+0000 7f815b7fe640 1 -- 172.21.15.3:0/3428457669 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c0aca40 con 0x7f815c0aa800 2024-05-21T17:42:34.544 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.541+0000 7f815b7fe640 1 --2- 172.21.15.3:0/3428457669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c0aa800 0x7f815c146aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f815c0ae4d0 tx=0x7f815002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe973bca7b7a784 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:34.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.541+0000 7f815affd640 1 -- 172.21.15.3:0/3428457669 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150034070 con 0x7f815c0aa800 2024-05-21T17:42:34.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.541+0000 7f815affd640 1 -- 172.21.15.3:0/3428457669 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150030860 con 0x7f815c0aa800 2024-05-21T17:42:34.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.541+0000 7f815affd640 1 -- 172.21.15.3:0/3428457669 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150030c30 con 0x7f815c0aa800 2024-05-21T17:42:34.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.542+0000 7f816248e640 1 -- 172.21.15.3:0/3428457669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c0aa800 msgr2=0x7f815c146aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.542+0000 7f816248e640 1 --2- 172.21.15.3:0/3428457669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c0aa800 0x7f815c146aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f815c0ae4d0 tx=0x7f815002b9a0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.545 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.542+0000 7f816248e640 1 -- 172.21.15.3:0/3428457669 shutdown_connections 2024-05-21T17:42:34.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.542+0000 7f816248e640 1 --2- 172.21.15.3:0/3428457669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c0aa800 0x7f815c146aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.542+0000 7f816248e640 1 --2- 172.21.15.3:0/3428457669 >> v2:172.21.15.203:3300/0 conn(0x7f815c0ac660 0x7f815c06c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.542+0000 7f816248e640 1 -- 172.21.15.3:0/3428457669 >> 172.21.15.3:0/3428457669 conn(0x7f815c0bbb90 msgr2=0x7f815c0bbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:34.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.542+0000 7f816248e640 1 -- 172.21.15.3:0/3428457669 shutdown_connections 2024-05-21T17:42:34.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.542+0000 7f816248e640 1 -- 172.21.15.3:0/3428457669 wait complete. 2024-05-21T17:42:34.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.543+0000 7f816248e640 1 Processor -- start 2024-05-21T17:42:34.546 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.543+0000 7f816248e640 1 -- start start 2024-05-21T17:42:34.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f816248e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f815c0aa800 0x7f815c1484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f816248e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f815c0ac660 0x7f815c13ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f816248e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c146fe0 0x7f815c145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f816248e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f815c0ae060 con 0x7f815c146fe0 2024-05-21T17:42:34.548 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f816248e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f815c0adee0 con 0x7f815c0aa800 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f816248e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f815c150c00 con 0x7f815c0ac660 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f8160a2b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c146fe0 0x7f815c145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f815b7fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f815c0ac660 0x7f815c13ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f815b7fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f815c0ac660 0x7f815c13ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:40852/0 (socket says 172.21.15.3:40852) 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f815b7fe640 1 -- 172.21.15.3:0/3279406029 learned_addr learned my addr 172.21.15.3:0/3279406029 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f8160a2b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c146fe0 0x7f815c145910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57846/0 (socket says 172.21.15.3:57846) 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f815bfff640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f815c0aa800 0x7f815c1484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f815b7fe640 1 -- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f815c0aa800 msgr2=0x7f815c1484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f815b7fe640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f815c0aa800 0x7f815c1484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f815b7fe640 1 -- 172.21.15.3:0/3279406029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c146fe0 msgr2=0x7f815c145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f815b7fe640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c146fe0 0x7f815c145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f815b7fe640 1 -- 172.21.15.3:0/3279406029 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c1499a0 con 0x7f815c0ac660 2024-05-21T17:42:34.549 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f8160a2b640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c146fe0 0x7f815c145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:34.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f815b7fe640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f815c0ac660 0x7f815c13ed10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8150000c00 tx=0x7f8150004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:34.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150048070 con 0x7f815c0ac660 2024-05-21T17:42:34.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150030660 con 0x7f815c0ac660 2024-05-21T17:42:34.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.544+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c149c60 con 0x7f815c0ac660 2024-05-21T17:42:34.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.545+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150034040 con 0x7f815c0ac660 2024-05-21T17:42:34.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.545+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c145e50 con 0x7f815c0ac660 2024-05-21T17:42:34.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.546+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815c069f80 con 0x7f815c0ac660 2024-05-21T17:42:34.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.547+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 5) ==== 146991+0+0 (secure 0 0 0) 0x7f815004b070 con 0x7f815c0ac660 2024-05-21T17:42:34.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.547+0000 7f8158ff9640 20 mgrc handle_mgr_map mgrmap(e 5) 2024-05-21T17:42:34.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.547+0000 7f8158ff9640 4 mgrc handle_mgr_map Got map version 5 2024-05-21T17:42:34.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.547+0000 7f8158ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:34.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.547+0000 7f8158ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:34.551 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.547+0000 7f8158ff9640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f81480b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:34.552 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.548+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4450+0+0 (secure 0 0 0) 0x7f81500fea40 con 0x7f815c0ac660 2024-05-21T17:42:34.552 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.548+0000 7f815bfff640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f81480b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:34.552 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.548+0000 7f815bfff640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f81480b16a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f814c006fd0 tx=0x7f814c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:34.553 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.552+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8150080990 con 0x7f815c0ac660 2024-05-21T17:42:34.794 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:34.792+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "mgr module enable", "module": "localpool"} v 0) -- 0x7f815c13fe50 con 0x7f815c0ac660 2024-05-21T17:42:35.343 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.341+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mgrmap(e 6) ==== 147004+0+0 (secure 0 0 0) 0x7f8150037070 con 0x7f815c0ac660 2024-05-21T17:42:35.344 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.341+0000 7f8158ff9640 20 mgrc handle_mgr_map mgrmap(e 6) 2024-05-21T17:42:35.344 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.342+0000 7f8158ff9640 4 mgrc handle_mgr_map Got map version 6 2024-05-21T17:42:35.344 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.342+0000 7f8158ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:35.344 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.343+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 <== mon.2 v2:172.21.15.3:3301/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "localpool"}]=0 v6) ==== 88+0+0 (secure 0 0 0) 0x7f81500ee020 con 0x7f815c0ac660 2024-05-21T17:42:35.364 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.362+0000 7f815bfff640 1 -- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 msgr2=0x7f81480b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-05-21T17:42:35.364 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.362+0000 7f815bfff640 1 -- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 msgr2=0x7f81480b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T17:42:35.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.362+0000 7f815bfff640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f81480b16a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f814c006fd0 tx=0x7f814c008040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T17:42:35.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.362+0000 7f815bfff640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f81480b16a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f814c006fd0 tx=0x7f814c008040 comp rx=0 tx=0).stop 2024-05-21T17:42:35.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.362+0000 7f8158ff9640 4 mgrc ms_handle_reset ms_handle_reset con 0x7f81480af1e0 2024-05-21T17:42:35.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.362+0000 7f8158ff9640 4 mgrc reconnect Terminating session with v2:172.21.15.203:6824/4157202067 2024-05-21T17:42:35.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.362+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 msgr2=0x7f81480b16a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-21T17:42:35.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.362+0000 7f8158ff9640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f81480b16a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:35.365 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.362+0000 7f8158ff9640 4 mgrc reconnect waiting to retry connect until 513.652406s 2024-05-21T17:42:35.461 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:35.459+0000 7f894be46040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T17:42:35.471 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:35.470+0000 7f3036643040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T17:42:35.471 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:35.470+0000 7feeada37040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T17:42:35.513 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.511+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "mgr module enable", "module": "localpool"} v 0) -- 0x7f81340050a0 con 0x7f815c0ac660 2024-05-21T17:42:35.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.547+0000 7f8142ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:35.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.547+0000 7f8142ffd640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f815c13f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:35.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.548+0000 7f815bfff640 1 -- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 msgr2=0x7f815c13f690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4157202067 2024-05-21T17:42:35.550 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.548+0000 7f815bfff640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f815c13f690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T17:42:35.580 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:35.578+0000 7f894be46040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:35.589 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:35.588+0000 7f3036643040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:35.590 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:35.588+0000 7feeada37040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:35.630 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:35.628+0000 7f894be46040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T17:42:35.638 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:35.637+0000 7feeada37040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T17:42:35.638 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:35.637+0000 7f3036643040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T17:42:35.751 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.749+0000 7f815bfff640 1 -- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 msgr2=0x7f815c13f690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4157202067 2024-05-21T17:42:35.751 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:35.749+0000 7f815bfff640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f815c13f690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T17:42:35.845 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:35.843+0000 7f894be46040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T17:42:35.853 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:35.852+0000 7f3036643040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T17:42:35.853 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:35.852+0000 7feeada37040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T17:42:35.890 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:35.888+0000 7f894be46040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T17:42:35.898 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:35.897+0000 7feeada37040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T17:42:35.899 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:35.897+0000 7f3036643040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T17:42:36.152 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.150+0000 7f815bfff640 1 -- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 msgr2=0x7f815c13f690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4157202067 2024-05-21T17:42:36.152 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.150+0000 7f815bfff640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f815c13f690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T17:42:36.345 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.342+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 <== mon.2 v2:172.21.15.3:3301/0 9 ==== mgrmap(e 7) ==== 147004+0+0 (secure 0 0 0) 0x7f81500e9b90 con 0x7f815c0ac660 2024-05-21T17:42:36.345 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.342+0000 7f8158ff9640 20 mgrc handle_mgr_map mgrmap(e 7) 2024-05-21T17:42:36.345 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.343+0000 7f8158ff9640 4 mgrc handle_mgr_map Got map version 7 2024-05-21T17:42:36.345 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.343+0000 7f8158ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:36.346 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.344+0000 7f8158ff9640 1 -- 172.21.15.3:0/3279406029 <== mon.2 v2:172.21.15.3:3301/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "localpool"}]=0 module 'localpool' is already enabled v7) ==== 125+0+0 (secure 0 0 0) 0x7f8150081a90 con 0x7f815c0ac660 2024-05-21T17:42:36.346 INFO:tasks.workunit.client.0.smithi003.stderr:module 'localpool' is already enabled 2024-05-21T17:42:36.349 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.346+0000 7f816248e640 10 mgrc shutdown 2024-05-21T17:42:36.349 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.346+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 msgr2=0x7f815c13f690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T17:42:36.349 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.346+0000 7f816248e640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f815c13f690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.349 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.347+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f815c0ac660 msgr2=0x7f815c13ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:36.349 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.347+0000 7f816248e640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f815c0ac660 0x7f815c13ed10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8150000c00 tx=0x7f8150004730 comp rx=0 tx=0).stop 2024-05-21T17:42:36.349 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.347+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 shutdown_connections 2024-05-21T17:42:36.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.347+0000 7f816248e640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7f81480af1e0 0x7f815c13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.347+0000 7f816248e640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f815c146fe0 0x7f815c145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.347+0000 7f816248e640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f815c0ac660 0x7f815c13ed10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.347+0000 7f816248e640 1 --2- 172.21.15.3:0/3279406029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f815c0aa800 0x7f815c1484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.347+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 >> 172.21.15.3:0/3279406029 conn(0x7f815c0bbb90 msgr2=0x7f815c0a8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:36.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.347+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 shutdown_connections 2024-05-21T17:42:36.350 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.347+0000 7f816248e640 1 -- 172.21.15.3:0/3279406029 wait complete. 2024-05-21T17:42:36.363 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool ls 2024-05-21T17:42:36.364 INFO:tasks.workunit.client.0.smithi003.stderr:+ grep '^by-host-' 2024-05-21T17:42:36.410 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:36.408+0000 7f894be46040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T17:42:36.412 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:36.411+0000 7feeada37040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T17:42:36.412 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:36.411+0000 7f3036643040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.444+0000 7feb692e6640 1 Processor -- start 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.444+0000 7feb692e6640 1 -- start start 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb692e6640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ac150 0x7feb640ac530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb692e6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feb640ad510 0x7feb640ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb692e6640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb641485e0 con 0x7feb640aaa30 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb692e6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7feb640aba30 con 0x7feb640ad510 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb692e6640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7feb64148080 con 0x7feb640ac150 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb62ffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ac150 0x7feb640ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb637fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7feb640aaa30 0x7feb64147970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44874/0 (socket says 172.21.15.3:44874) 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb62ffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ac150 0x7feb640ac530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42444/0 (socket says 172.21.15.3:42444) 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb637fe640 1 -- 172.21.15.3:0/719121995 learned_addr learned my addr 172.21.15.3:0/719121995 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:42:36.448 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb627fc640 1 --2- 172.21.15.3:0/719121995 >> v2:172.21.15.203:3300/0 conn(0x7feb640ad510 0x7feb640ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:36.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb61ffb640 1 -- 172.21.15.3:0/719121995 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531532206 0 0) 0x7feb641485e0 con 0x7feb640aaa30 2024-05-21T17:42:36.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb62ffd640 1 -- 172.21.15.3:0/719121995 >> v2:172.21.15.203:3300/0 conn(0x7feb640ad510 msgr2=0x7feb640ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:36.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb62ffd640 1 --2- 172.21.15.3:0/719121995 >> v2:172.21.15.203:3300/0 conn(0x7feb640ad510 0x7feb640ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb62ffd640 1 -- 172.21.15.3:0/719121995 >> v1:172.21.15.3:6789/0 conn(0x7feb640aaa30 legacy=0x7feb64147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:36.449 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.445+0000 7feb62ffd640 1 -- 172.21.15.3:0/719121995 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb64148250 con 0x7feb640ac150 2024-05-21T17:42:36.450 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.446+0000 7feb62ffd640 1 --2- 172.21.15.3:0/719121995 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ac150 0x7feb640ac530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0097a0 tx=0x7feb4c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b6c857635c105d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:36.450 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.446+0000 7feb61ffb640 1 -- 172.21.15.3:0/719121995 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c0315a0 con 0x7feb640ac150 2024-05-21T17:42:36.450 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.446+0000 7feb61ffb640 1 -- 172.21.15.3:0/719121995 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb4c037070 con 0x7feb640ac150 2024-05-21T17:42:36.450 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.446+0000 7feb61ffb640 1 -- 172.21.15.3:0/719121995 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c031960 con 0x7feb640ac150 2024-05-21T17:42:36.451 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.447+0000 7feb692e6640 1 -- 172.21.15.3:0/719121995 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ac150 msgr2=0x7feb640ac530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:36.451 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.447+0000 7feb692e6640 1 --2- 172.21.15.3:0/719121995 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ac150 0x7feb640ac530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0097a0 tx=0x7feb4c02b840 comp rx=0 tx=0).stop 2024-05-21T17:42:36.451 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.447+0000 7feb692e6640 1 -- 172.21.15.3:0/719121995 shutdown_connections 2024-05-21T17:42:36.451 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.447+0000 7feb692e6640 1 --2- 172.21.15.3:0/719121995 >> v2:172.21.15.203:3300/0 conn(0x7feb640ad510 0x7feb640ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.451 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.447+0000 7feb692e6640 1 --2- 172.21.15.3:0/719121995 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ac150 0x7feb640ac530 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.451 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.447+0000 7feb692e6640 1 -- 172.21.15.3:0/719121995 >> 172.21.15.3:0/719121995 conn(0x7feb6406c830 msgr2=0x7feb6406cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:36.451 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.447+0000 7feb692e6640 1 -- 172.21.15.3:0/719121995 shutdown_connections 2024-05-21T17:42:36.451 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.447+0000 7feb692e6640 1 -- 172.21.15.3:0/719121995 wait complete. 2024-05-21T17:42:36.451 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.448+0000 7feb692e6640 1 Processor -- start 2024-05-21T17:42:36.451 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.448+0000 7feb692e6640 1 -- start start 2024-05-21T17:42:36.453 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb692e6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb640aaa30 0x7feb6409dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:36.453 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb692e6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb640ac150 0x7feb641db780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:36.453 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb692e6640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ad510 0x7feb6409c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:36.453 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb692e6640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feb640aafd0 con 0x7feb640ac150 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb692e6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb640aae50 con 0x7feb640aaa30 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb692e6640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7feb640ab150 con 0x7feb640ad510 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb627fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb640ac150 0x7feb641db780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb637fe640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ad510 0x7feb6409c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb627fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb640ac150 0x7feb641db780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37816/0 (socket says 172.21.15.3:37816) 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb627fc640 1 -- 172.21.15.3:0/4190778831 learned_addr learned my addr 172.21.15.3:0/4190778831 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb62ffd640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb640aaa30 0x7feb6409dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb627fc640 1 -- 172.21.15.3:0/4190778831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ad510 msgr2=0x7feb6409c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb627fc640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ad510 0x7feb6409c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb627fc640 1 -- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb640aaa30 msgr2=0x7feb6409dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb627fc640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb640aaa30 0x7feb6409dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.454 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.449+0000 7feb627fc640 1 -- 172.21.15.3:0/4190778831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb641dbf50 con 0x7feb640ac150 2024-05-21T17:42:36.455 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.450+0000 7feb637fe640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ad510 0x7feb6409c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:36.455 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.450+0000 7feb627fc640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb640ac150 0x7feb641db780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feb5400e870 tx=0x7feb5400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:36.455 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.450+0000 7feb5b7fe640 1 -- 172.21.15.3:0/4190778831 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54017070 con 0x7feb640ac150 2024-05-21T17:42:36.455 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.450+0000 7feb5b7fe640 1 -- 172.21.15.3:0/4190778831 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5400c940 con 0x7feb640ac150 2024-05-21T17:42:36.455 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.450+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb641dc210 con 0x7feb640ac150 2024-05-21T17:42:36.455 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.450+0000 7feb5b7fe640 1 -- 172.21.15.3:0/4190778831 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5400cd30 con 0x7feb640ac150 2024-05-21T17:42:36.455 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.450+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb641df2b0 con 0x7feb640ac150 2024-05-21T17:42:36.455 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.451+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb640507e0 con 0x7feb640ac150 2024-05-21T17:42:36.456 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.452+0000 7feb5b7fe640 1 -- 172.21.15.3:0/4190778831 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 147004+0+0 (secure 0 0 0) 0x7feb5401a070 con 0x7feb640ac150 2024-05-21T17:42:36.456 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.452+0000 7feb5b7fe640 20 mgrc handle_mgr_map mgrmap(e 7) 2024-05-21T17:42:36.456 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.453+0000 7feb5b7fe640 4 mgrc handle_mgr_map Got map version 7 2024-05-21T17:42:36.456 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.453+0000 7feb5b7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:36.456 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.453+0000 7feb5b7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] 2024-05-21T17:42:36.456 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.453+0000 7feb5b7fe640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7feb440af2b0 0x7feb440b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:36.456 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.453+0000 7feb62ffd640 1 -- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7feb440af2b0 msgr2=0x7feb440b1770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4157202067 2024-05-21T17:42:36.456 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.453+0000 7feb5b7fe640 1 -- 172.21.15.3:0/4190778831 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4450+0+0 (secure 0 0 0) 0x7feb540dcfa0 con 0x7feb640ac150 2024-05-21T17:42:36.456 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.453+0000 7feb62ffd640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7feb440af2b0 0x7feb440b1770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T17:42:36.462 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.461+0000 7feb5b7fe640 1 -- 172.21.15.3:0/4190778831 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feb540a6680 con 0x7feb640ac150 2024-05-21T17:42:36.500 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:36.499+0000 7feeada37040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:36.500 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:36.499+0000 7f3036643040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:36.506 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:36.504+0000 7f894be46040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:36.543 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:36.542+0000 7feeada37040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T17:42:36.543 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:36.542+0000 7f3036643040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T17:42:36.551 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:36.549+0000 7f894be46040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T17:42:36.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.651+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) -- 0x7feb640aa080 con 0x7feb640ac150 2024-05-21T17:42:36.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.651+0000 7feb5b7fe640 1 -- 172.21.15.3:0/4190778831 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v18) ==== 59+0+37 (secure 0 0 0) 0x7feb540aa980 con 0x7feb640ac150 2024-05-21T17:42:36.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.654+0000 7feb62ffd640 1 -- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7feb440af2b0 msgr2=0x7feb440b1770 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4157202067 2024-05-21T17:42:36.656 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.654+0000 7feb62ffd640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7feb440af2b0 0x7feb440b1770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T17:42:36.711 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:36.710+0000 7feeada37040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T17:42:36.711 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:36.710+0000 7f3036643040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T17:42:36.724 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:36.722+0000 7f894be46040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T17:42:36.756 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:36.756+0000 7feeada37040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T17:42:36.757 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:36.756+0000 7f3036643040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T17:42:36.766 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.764+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) -- 0x7feb640a9dc0 con 0x7feb640ac150 2024-05-21T17:42:36.766 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.764+0000 7feb5b7fe640 1 -- 172.21.15.3:0/4190778831 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v18) ==== 59+0+37 (secure 0 0 0) 0x7feb540af430 con 0x7feb640ac150 2024-05-21T17:42:36.769 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 10 mgrc shutdown 2024-05-21T17:42:36.769 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7feb440af2b0 msgr2=0x7feb440b1770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T17:42:36.769 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7feb440af2b0 0x7feb440b1770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.769 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb640ac150 msgr2=0x7feb641db780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:36.769 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb640ac150 0x7feb641db780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feb5400e870 tx=0x7feb5400ed30 comp rx=0 tx=0).stop 2024-05-21T17:42:36.769 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 shutdown_connections 2024-05-21T17:42:36.769 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:6824/4157202067,v1:172.21.15.203:6825/4157202067] conn(0x7feb440af2b0 0x7feb440b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.769 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7feb640ad510 0x7feb6409c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.770 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feb640ac150 0x7feb641db780 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.770 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 1 --2- 172.21.15.3:0/4190778831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb640aaa30 0x7feb6409dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:36.770 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.766+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 >> 172.21.15.3:0/4190778831 conn(0x7feb6406c830 msgr2=0x7feb641cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:36.770 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.767+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 shutdown_connections 2024-05-21T17:42:36.770 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:36.767+0000 7feb692e6640 1 -- 172.21.15.3:0/4190778831 wait complete. 2024-05-21T17:42:36.772 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:36.771+0000 7f894be46040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T17:42:36.778 INFO:tasks.workunit.client.0.smithi003.stderr:+ sleep 5 2024-05-21T17:42:36.807 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:36.806+0000 7feeada37040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T17:42:36.808 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:36.807+0000 7f3036643040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T17:42:36.824 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:36.822+0000 7f894be46040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T17:42:36.922 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:36.921+0000 7f3036643040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:36.922 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:36.921+0000 7feeada37040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:36.941 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:36.939+0000 7f894be46040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:37.258 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:37.257+0000 7feeada37040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T17:42:37.259 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:37.258+0000 7f3036643040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T17:42:37.280 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:37.278+0000 7f894be46040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T17:42:37.385 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:37.384+0000 7feeada37040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T17:42:37.386 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:37.385+0000 7f3036643040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T17:42:37.411 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:37.409+0000 7f894be46040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T17:42:37.603 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:37.602+0000 7feeada37040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T17:42:37.604 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:37.603+0000 7f3036643040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T17:42:37.633 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:37.631+0000 7f894be46040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T17:42:37.654 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:37.653+0000 7feeada37040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T17:42:37.655 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:37.655+0000 7f3036643040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T17:42:37.685 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:37.683+0000 7f894be46040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T17:42:37.770 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:37.769+0000 7feeada37040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T17:42:37.771 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:37.770+0000 7f3036643040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T17:42:37.805 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:37.803+0000 7f894be46040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T17:42:38.041 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:38.040+0000 7feeada37040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T17:42:38.044 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:38.043+0000 7f3036643040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T17:42:38.086 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:38.085+0000 7feeada37040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T17:42:38.088 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:38.086+0000 7f894be46040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T17:42:38.089 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:38.088+0000 7f3036643040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T17:42:38.134 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:38.132+0000 7f894be46040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T17:42:38.235 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:38.233+0000 7feeada37040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T17:42:38.238 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:38.237+0000 7f3036643040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T17:42:38.285 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:38.283+0000 7f894be46040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T17:42:38.371 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:38.370+0000 7feeada37040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T17:42:38.373 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:38.372+0000 7f3036643040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T17:42:38.421 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:38.420+0000 7feeada37040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T17:42:38.424 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:38.423+0000 7f3036643040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T17:42:38.425 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:38.423+0000 7f894be46040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T17:42:38.476 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:38.474+0000 7f894be46040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T17:42:38.477 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:38.475+0000 7feeada37040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:38.479 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:38.478+0000 7f3036643040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:38.521 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:38.520+0000 7feeada37040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T17:42:38.524 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:38.523+0000 7f3036643040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T17:42:38.533 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:38.531+0000 7f894be46040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:38.578 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:38.576+0000 7f894be46040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T17:42:38.924 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:38.923+0000 7feeada37040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T17:42:38.925 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:38.924+0000 7f3036643040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T17:42:38.987 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:38.985+0000 7f894be46040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T17:42:39.044 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.043+0000 7feeada37040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T17:42:39.044 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:39.044+0000 7f3036643040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T17:42:39.105 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.103+0000 7fee2e555640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:39.105 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.103+0000 7fee2e555640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:39.105 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.103+0000 7fee2e555640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:39.105 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.103+0000 7fee2e555640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:39.105 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.103+0000 7fee2e555640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:39.105 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.104+0000 7fee3155b640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:39.106 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.104+0000 7fee3155b640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:39.106 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.104+0000 7fee3155b640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:39.106 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.104+0000 7fee3155b640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:39.106 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:39.104+0000 7fee3155b640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:39.117 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:39.115+0000 7f894be46040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T17:42:41.780 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool ls 2024-05-21T17:42:41.780 INFO:tasks.workunit.client.0.smithi003.stderr:+ grep '^by-host-' 2024-05-21T17:42:41.871 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.869+0000 7fcaec2bc640 1 Processor -- start 2024-05-21T17:42:41.871 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.869+0000 7fcaec2bc640 1 -- start start 2024-05-21T17:42:41.872 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaec2bc640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcae4145cf0 0x7fcae40a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:41.872 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaec2bc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcae40a68f0 0x7fcae4150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:41.872 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaec2bc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcae40a85b0 con 0x7fcae40a72d0 2024-05-21T17:42:41.872 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaec2bc640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fcae40738b0 con 0x7fcae4145cf0 2024-05-21T17:42:41.873 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaec2bc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fcae41460d0 con 0x7fcae40a68f0 2024-05-21T17:42:41.873 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaea058640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcae4145cf0 0x7fcae40a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:41.873 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaea058640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fcae4145cf0 0x7fcae40a6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37826/0 (socket says 172.21.15.3:37826) 2024-05-21T17:42:41.873 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaea058640 1 -- 172.21.15.3:0/3591121041 learned_addr learned my addr 172.21.15.3:0/3591121041 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:41.873 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaea859640 1 --2- 172.21.15.3:0/3591121041 >> v2:172.21.15.203:3300/0 conn(0x7fcae40a68f0 0x7fcae4150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:41.873 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaea058640 1 -- 172.21.15.3:0/3591121041 >> v2:172.21.15.203:3300/0 conn(0x7fcae40a68f0 msgr2=0x7fcae4150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:41.873 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaea058640 1 --2- 172.21.15.3:0/3591121041 >> v2:172.21.15.203:3300/0 conn(0x7fcae40a68f0 0x7fcae4150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:41.873 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaea058640 1 -- 172.21.15.3:0/3591121041 >> v1:172.21.15.203:6789/0 conn(0x7fcae40a72d0 legacy=0x7fcae40a8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:41.873 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.870+0000 7fcaea058640 1 -- 172.21.15.3:0/3591121041 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae406d570 con 0x7fcae4145cf0 2024-05-21T17:42:41.874 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.871+0000 7fcaea058640 1 --2- 172.21.15.3:0/3591121041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae4145cf0 0x7fcae40a6d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcae4073120 tx=0x7fcae002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d2885ce8a46397 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:41.874 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.871+0000 7fcae9056640 1 -- 172.21.15.3:0/3591121041 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0038070 con 0x7fcae4145cf0 2024-05-21T17:42:41.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.871+0000 7fcae9056640 1 -- 172.21.15.3:0/3591121041 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae002f050 con 0x7fcae4145cf0 2024-05-21T17:42:41.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.871+0000 7fcae9056640 1 -- 172.21.15.3:0/3591121041 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0031970 con 0x7fcae4145cf0 2024-05-21T17:42:41.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.872+0000 7fcaec2bc640 1 -- 172.21.15.3:0/3591121041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae4145cf0 msgr2=0x7fcae40a6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:41.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.872+0000 7fcaec2bc640 1 --2- 172.21.15.3:0/3591121041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae4145cf0 0x7fcae40a6d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcae4073120 tx=0x7fcae002b7f0 comp rx=0 tx=0).stop 2024-05-21T17:42:41.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.872+0000 7fcaec2bc640 1 -- 172.21.15.3:0/3591121041 shutdown_connections 2024-05-21T17:42:41.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.872+0000 7fcaec2bc640 1 --2- 172.21.15.3:0/3591121041 >> v2:172.21.15.203:3300/0 conn(0x7fcae40a68f0 0x7fcae4150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:41.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.872+0000 7fcaec2bc640 1 --2- 172.21.15.3:0/3591121041 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae4145cf0 0x7fcae40a6d90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:41.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.872+0000 7fcaec2bc640 1 -- 172.21.15.3:0/3591121041 >> 172.21.15.3:0/3591121041 conn(0x7fcae4072600 msgr2=0x7fcae4075420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:41.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.872+0000 7fcaec2bc640 1 -- 172.21.15.3:0/3591121041 shutdown_connections 2024-05-21T17:42:41.875 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.872+0000 7fcaec2bc640 1 -- 172.21.15.3:0/3591121041 wait complete. 2024-05-21T17:42:41.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.873+0000 7fcaec2bc640 1 Processor -- start 2024-05-21T17:42:41.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.873+0000 7fcaec2bc640 1 -- start start 2024-05-21T17:42:41.877 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcaec2bc640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcae40a68f0 0x7fcae41d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcaec2bc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae40a72d0 0x7fcae41db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcaec2bc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae4145cf0 0x7fcae41d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcaec2bc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcae40a9100 con 0x7fcae40a72d0 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcaec2bc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcae40a8f80 con 0x7fcae4145cf0 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcaec2bc640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fcae406f2f0 con 0x7fcae40a68f0 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcae9857640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae40a72d0 0x7fcae41db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcaea058640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcae40a68f0 0x7fcae41d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcae9857640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae40a72d0 0x7fcae41db7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37842/0 (socket says 172.21.15.3:37842) 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcaea058640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcae40a68f0 0x7fcae41d2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42458/0 (socket says 172.21.15.3:42458) 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcae9857640 1 -- 172.21.15.3:0/4159527386 learned_addr learned my addr 172.21.15.3:0/4159527386 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcaea859640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae4145cf0 0x7fcae41d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcae9857640 1 -- 172.21.15.3:0/4159527386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcae40a68f0 msgr2=0x7fcae41d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:41.878 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcae9857640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcae40a68f0 0x7fcae41d2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:41.879 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcae9857640 1 -- 172.21.15.3:0/4159527386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae4145cf0 msgr2=0x7fcae41d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:41.879 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcae9857640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae4145cf0 0x7fcae41d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:41.879 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcae9857640 1 -- 172.21.15.3:0/4159527386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae41dc120 con 0x7fcae40a72d0 2024-05-21T17:42:41.879 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcae9857640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae40a72d0 0x7fcae41db7f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcad400c460 tx=0x7fcad400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:41.879 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcadaffd640 1 -- 172.21.15.3:0/4159527386 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4016020 con 0x7fcae40a72d0 2024-05-21T17:42:41.879 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcadaffd640 1 -- 172.21.15.3:0/4159527386 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad4012870 con 0x7fcae40a72d0 2024-05-21T17:42:41.879 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae41dc3e0 con 0x7fcae40a72d0 2024-05-21T17:42:41.879 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.874+0000 7fcadaffd640 1 -- 172.21.15.3:0/4159527386 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4012c60 con 0x7fcae40a72d0 2024-05-21T17:42:41.879 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.875+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae41d2630 con 0x7fcae40a72d0 2024-05-21T17:42:41.879 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.875+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae40507e0 con 0x7fcae40a72d0 2024-05-21T17:42:41.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.877+0000 7fcadaffd640 1 -- 172.21.15.3:0/4159527386 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) ==== 147050+0+0 (secure 0 0 0) 0x7fcad4019020 con 0x7fcae40a72d0 2024-05-21T17:42:41.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.877+0000 7fcadaffd640 20 mgrc handle_mgr_map mgrmap(e 10) 2024-05-21T17:42:41.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.878+0000 7fcadaffd640 4 mgrc handle_mgr_map Got map version 10 2024-05-21T17:42:41.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.878+0000 7fcadaffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:41.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.878+0000 7fcadaffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:41.880 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.878+0000 7fcadaffd640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fcac80af340 0x7fcac80b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:41.881 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.878+0000 7fcadaffd640 1 -- 172.21.15.3:0/4159527386 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5137+0+0 (secure 0 0 0) 0x7fcad40dce50 con 0x7fcae40a72d0 2024-05-21T17:42:41.881 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.878+0000 7fcaea058640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fcac80af340 0x7fcac80b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:41.881 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.879+0000 7fcaea058640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fcac80af340 0x7fcac80b1800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcae0035000 tx=0x7fcae0008030 comp rx=0 tx=0).ready entity=mgr.4435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:41.884 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:41.882+0000 7fcadaffd640 1 -- 172.21.15.3:0/4159527386 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcad4020070 con 0x7fcae40a72d0 2024-05-21T17:42:42.075 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.073+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) -- 0x7fcae4073410 con 0x7fcae40a72d0 2024-05-21T17:42:42.075 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.073+0000 7fcadaffd640 1 -- 172.21.15.3:0/4159527386 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v21) ==== 59+0+55 (secure 0 0 0) 0x7fcad40a68e0 con 0x7fcae40a72d0 2024-05-21T17:42:42.210 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.208+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) -- 0x7fcae4141370 con 0x7fcae40a72d0 2024-05-21T17:42:42.210 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.208+0000 7fcadaffd640 1 -- 172.21.15.3:0/4159527386 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v22) ==== 59+0+55 (secure 0 0 0) 0x7fcad40a61e0 con 0x7fcae40a72d0 2024-05-21T17:42:42.213 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.210+0000 7fcaec2bc640 10 mgrc shutdown 2024-05-21T17:42:42.213 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.210+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fcac80af340 msgr2=0x7fcac80b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:42.213 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.210+0000 7fcaec2bc640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fcac80af340 0x7fcac80b1800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcae0035000 tx=0x7fcae0008030 comp rx=0 tx=0).stop 2024-05-21T17:42:42.213 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.211+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae40a72d0 msgr2=0x7fcae41db7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:42.213 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.211+0000 7fcaec2bc640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae40a72d0 0x7fcae41db7f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcad400c460 tx=0x7fcad400c920 comp rx=0 tx=0).stop 2024-05-21T17:42:42.213 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.211+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 shutdown_connections 2024-05-21T17:42:42.214 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.211+0000 7fcaec2bc640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fcac80af340 0x7fcac80b1800 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:42.214 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.211+0000 7fcaec2bc640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae4145cf0 0x7fcae41d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:42.214 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.211+0000 7fcaec2bc640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcae40a72d0 0x7fcae41db7f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:42.214 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.211+0000 7fcaec2bc640 1 --2- 172.21.15.3:0/4159527386 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fcae40a68f0 0x7fcae41d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:42.214 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.211+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 >> 172.21.15.3:0/4159527386 conn(0x7fcae4072600 msgr2=0x7fcae4143d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:42.214 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.211+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 shutdown_connections 2024-05-21T17:42:42.214 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.211+0000 7fcaec2bc640 1 -- 172.21.15.3:0/4159527386 wait complete. 2024-05-21T17:42:42.222 INFO:tasks.workunit.client.0.smithi003.stdout:by-host-smithi003 2024-05-21T17:42:42.222 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph mgr module disable localpool 2024-05-21T17:42:42.304 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.302+0000 7fd84fe2a640 1 Processor -- start 2024-05-21T17:42:42.305 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.302+0000 7fd84fe2a640 1 -- start start 2024-05-21T17:42:42.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.303+0000 7fd84fe2a640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fd8480aa800 0x7fd84806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:42.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.303+0000 7fd84fe2a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd8480ad7d0 0x7fd8480adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:42.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.303+0000 7fd84fe2a640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8480ae130 con 0x7fd8480ac660 2024-05-21T17:42:42.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.303+0000 7fd84fe2a640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fd8480ac210 con 0x7fd8480aa800 2024-05-21T17:42:42.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.303+0000 7fd84fe2a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fd848146670 con 0x7fd8480ad7d0 2024-05-21T17:42:42.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.303+0000 7fd84dbc6640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd8480ac660 0x7fd848146aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44882/0 (socket says 172.21.15.3:44882) 2024-05-21T17:42:42.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.303+0000 7fd84dbc6640 1 -- 172.21.15.3:0/3466232338 learned_addr learned my addr 172.21.15.3:0/3466232338 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:42:42.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.303+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/3466232338 >> v2:172.21.15.3:3300/0 conn(0x7fd8480aa800 0x7fd84806c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:42.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.304+0000 7fd84e3c7640 1 --2- 172.21.15.3:0/3466232338 >> v2:172.21.15.203:3300/0 conn(0x7fd8480ad7d0 0x7fd8480adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:42.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.304+0000 7fd84d3c5640 1 -- 172.21.15.3:0/3466232338 >> v2:172.21.15.203:3300/0 conn(0x7fd8480ad7d0 msgr2=0x7fd8480adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:42.307 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.304+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/3466232338 >> v2:172.21.15.203:3300/0 conn(0x7fd8480ad7d0 0x7fd8480adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:42.307 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.304+0000 7fd84d3c5640 1 -- 172.21.15.3:0/3466232338 >> v1:172.21.15.3:6789/0 conn(0x7fd8480ac660 legacy=0x7fd848146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:42.307 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.304+0000 7fd84d3c5640 1 -- 172.21.15.3:0/3466232338 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd848146360 con 0x7fd8480aa800 2024-05-21T17:42:42.307 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.304+0000 7fd84cbc4640 1 -- 172.21.15.3:0/3466232338 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1948962043 0 0) 0x7fd8480ae130 con 0x7fd8480ac660 2024-05-21T17:42:42.308 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.304+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/3466232338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 0x7fd84806c6f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd84806d4b0 tx=0x7fd83802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f540a8707ae60fb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:42.308 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.304+0000 7fd84cbc4640 1 -- 172.21.15.3:0/3466232338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838035020 con 0x7fd8480aa800 2024-05-21T17:42:42.308 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.304+0000 7fd84cbc4640 1 -- 172.21.15.3:0/3466232338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd838031830 con 0x7fd8480aa800 2024-05-21T17:42:42.308 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.304+0000 7fd84cbc4640 1 -- 172.21.15.3:0/3466232338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838031c00 con 0x7fd8480aa800 2024-05-21T17:42:42.308 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.305+0000 7fd84fe2a640 1 -- 172.21.15.3:0/3466232338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 msgr2=0x7fd84806c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:42.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.305+0000 7fd84fe2a640 1 --2- 172.21.15.3:0/3466232338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 0x7fd84806c6f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd84806d4b0 tx=0x7fd83802b8d0 comp rx=0 tx=0).stop 2024-05-21T17:42:42.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.305+0000 7fd84fe2a640 1 -- 172.21.15.3:0/3466232338 shutdown_connections 2024-05-21T17:42:42.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.305+0000 7fd84fe2a640 1 --2- 172.21.15.3:0/3466232338 >> v2:172.21.15.203:3300/0 conn(0x7fd8480ad7d0 0x7fd8480adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:42.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.305+0000 7fd84fe2a640 1 --2- 172.21.15.3:0/3466232338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 0x7fd84806c6f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:42.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.305+0000 7fd84fe2a640 1 -- 172.21.15.3:0/3466232338 >> 172.21.15.3:0/3466232338 conn(0x7fd8480bbb90 msgr2=0x7fd8480bbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:42.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.305+0000 7fd84fe2a640 1 -- 172.21.15.3:0/3466232338 shutdown_connections 2024-05-21T17:42:42.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.305+0000 7fd84fe2a640 1 -- 172.21.15.3:0/3466232338 wait complete. 2024-05-21T17:42:42.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.306+0000 7fd84fe2a640 1 Processor -- start 2024-05-21T17:42:42.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.306+0000 7fd84fe2a640 1 -- start start 2024-05-21T17:42:42.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84fe2a640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 0x7fd84814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:42.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84fe2a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480ac660 0x7fd84814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:42.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84fe2a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8480ad7d0 0x7fd848140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:42.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84fe2a640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd8481471b0 con 0x7fd8480aa800 2024-05-21T17:42:42.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84fe2a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8480acb30 con 0x7fd8480ac660 2024-05-21T17:42:42.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84fe2a640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fd84814c020 con 0x7fd8480ad7d0 2024-05-21T17:42:42.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84dbc6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 0x7fd84814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:42.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84dbc6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 0x7fd84814b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37846/0 (socket says 172.21.15.3:37846) 2024-05-21T17:42:42.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84dbc6640 1 -- 172.21.15.3:0/1148714844 learned_addr learned my addr 172.21.15.3:0/1148714844 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:42.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84e3c7640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8480ad7d0 0x7fd848140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:42.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480ac660 0x7fd84814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:42.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84dbc6640 1 -- 172.21.15.3:0/1148714844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8480ad7d0 msgr2=0x7fd848140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84dbc6640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8480ad7d0 0x7fd848140aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84dbc6640 1 -- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480ac660 msgr2=0x7fd84814ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84dbc6640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480ac660 0x7fd84814ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84dbc6640 1 -- 172.21.15.3:0/1148714844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84814dfb0 con 0x7fd8480aa800 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84dbc6640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 0x7fd84814b2b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c490 tx=0x7fd84400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd83e7fc640 1 -- 172.21.15.3:0/1148714844 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844018070 con 0x7fd8480aa800 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd83e7fc640 1 -- 172.21.15.3:0/1148714844 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd844007500 con 0x7fd8480aa800 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.307+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84814e290 con 0x7fd8480aa800 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.308+0000 7fd83e7fc640 1 -- 172.21.15.3:0/1148714844 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844004a20 con 0x7fd8480aa800 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.308+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84814ef60 con 0x7fd8480aa800 2024-05-21T17:42:42.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.309+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd848069f80 con 0x7fd8480aa800 2024-05-21T17:42:42.314 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.310+0000 7fd83e7fc640 1 -- 172.21.15.3:0/1148714844 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 10) ==== 147050+0+0 (secure 0 0 0) 0x7fd84401b070 con 0x7fd8480aa800 2024-05-21T17:42:42.314 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.310+0000 7fd83e7fc640 20 mgrc handle_mgr_map mgrmap(e 10) 2024-05-21T17:42:42.314 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.310+0000 7fd83e7fc640 4 mgrc handle_mgr_map Got map version 10 2024-05-21T17:42:42.314 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.310+0000 7fd83e7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:42.314 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.310+0000 7fd83e7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:42.315 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.310+0000 7fd83e7fc640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd82c0b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:42.315 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.311+0000 7fd83e7fc640 1 -- 172.21.15.3:0/1148714844 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5137+0+0 (secure 0 0 0) 0x7fd8440dbf20 con 0x7fd8480aa800 2024-05-21T17:42:42.315 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.311+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd82c0b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:42.315 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.311+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd82c0b1800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd838036000 tx=0x7fd838008030 comp rx=0 tx=0).ready entity=mgr.4435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:42.316 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.315+0000 7fd83e7fc640 1 -- 172.21.15.3:0/1148714844 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd8440a5360 con 0x7fd8480aa800 2024-05-21T17:42:42.557 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:42.555+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "localpool"} v 0) -- 0x7fd8480bb7f0 con 0x7fd8480aa800 2024-05-21T17:42:43.128 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.126+0000 7fd83e7fc640 1 -- 172.21.15.3:0/1148714844 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "localpool"}]=0 v11) ==== 89+0+0 (secure 0 0 0) 0x7fd8440a9660 con 0x7fd8480aa800 2024-05-21T17:42:43.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.134+0000 7fd83e7fc640 1 -- 172.21.15.3:0/1148714844 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 11) ==== 147093+0+0 (secure 0 0 0) 0x7fd8440a1560 con 0x7fd8480aa800 2024-05-21T17:42:43.136 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.134+0000 7fd83e7fc640 20 mgrc handle_mgr_map mgrmap(e 11) 2024-05-21T17:42:43.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.135+0000 7fd83e7fc640 4 mgrc handle_mgr_map Got map version 11 2024-05-21T17:42:43.137 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.135+0000 7fd83e7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:43.163 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.161+0000 7fd84d3c5640 1 -- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 msgr2=0x7fd82c0b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-21T17:42:43.163 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.161+0000 7fd84d3c5640 1 -- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 msgr2=0x7fd82c0b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-21T17:42:43.163 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.161+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd82c0b1800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd838036000 tx=0x7fd838008030 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-21T17:42:43.163 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.161+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd82c0b1800 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd838036000 tx=0x7fd838008030 comp rx=0 tx=0).stop 2024-05-21T17:42:43.163 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.161+0000 7fd83e7fc640 4 mgrc ms_handle_reset ms_handle_reset con 0x7fd82c0af340 2024-05-21T17:42:43.163 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.161+0000 7fd83e7fc640 4 mgrc reconnect Terminating session with v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:43.163 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.161+0000 7fd83e7fc640 1 -- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 msgr2=0x7fd82c0b1800 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-21T17:42:43.163 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.161+0000 7fd83e7fc640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd82c0b1800 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:43.163 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.161+0000 7fd83e7fc640 4 mgrc reconnect waiting to retry connect until 521.415463s 2024-05-21T17:42:43.252 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:43.250+0000 7f9dbba13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T17:42:43.262 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:43.261+0000 7f502964d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T17:42:43.263 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:43.262+0000 7fc84c213040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-21T17:42:43.297 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.296+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "localpool"} v 0) -- 0x7fd8240050a0 con 0x7fd8480aa800 2024-05-21T17:42:43.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.311+0000 7fd83cff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:43.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.311+0000 7fd83cff9640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd848140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:43.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.311+0000 7fd84d3c5640 1 -- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 msgr2=0x7fd848140fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:43.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.311+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd848140fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T17:42:43.370 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:43.368+0000 7f9dbba13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:43.383 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:43.382+0000 7fc84c213040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:43.383 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:43.382+0000 7f502964d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:43.419 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:43.418+0000 7f9dbba13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T17:42:43.434 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:43.431+0000 7fc84c213040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T17:42:43.434 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:43.431+0000 7f502964d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-21T17:42:43.514 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.512+0000 7fd84d3c5640 1 -- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 msgr2=0x7fd848140fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:43.514 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.512+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd848140fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T17:42:43.634 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:43.632+0000 7f9dbba13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T17:42:43.647 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:43.646+0000 7f502964d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T17:42:43.647 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:43.646+0000 7fc84c213040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-21T17:42:43.679 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:43.678+0000 7f9dbba13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T17:42:43.692 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:43.691+0000 7f502964d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T17:42:43.692 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:43.691+0000 7fc84c213040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-21T17:42:43.915 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.913+0000 7fd84d3c5640 1 -- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 msgr2=0x7fd848140fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:43.915 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:43.913+0000 7fd84d3c5640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd848140fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T17:42:44.130 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.128+0000 7fd83e7fc640 1 -- 172.21.15.3:0/1148714844 <== mon.0 v2:172.21.15.3:3300/0 9 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "localpool"}]=0 module 'localpool' is already disabled v12) ==== 127+0+0 (secure 0 0 0) 0x7fd8440981a0 con 0x7fd8480aa800 2024-05-21T17:42:44.131 INFO:tasks.workunit.client.0.smithi003.stderr:module 'localpool' is already disabled 2024-05-21T17:42:44.133 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 10 mgrc shutdown 2024-05-21T17:42:44.133 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 msgr2=0x7fd848140fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T17:42:44.133 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd848140fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.133 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 msgr2=0x7fd84814b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.133 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 0x7fd84814b2b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c490 tx=0x7fd84400c950 comp rx=0 tx=0).stop 2024-05-21T17:42:44.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 shutdown_connections 2024-05-21T17:42:44.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fd82c0af340 0x7fd848140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd8480ad7d0 0x7fd848140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.134 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8480ac660 0x7fd84814ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.135 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 1 --2- 172.21.15.3:0/1148714844 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd8480aa800 0x7fd84814b2b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.135 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.131+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 >> 172.21.15.3:0/1148714844 conn(0x7fd8480bbb90 msgr2=0x7fd8480adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:44.135 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.132+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 shutdown_connections 2024-05-21T17:42:44.135 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.133+0000 7fd84fe2a640 1 -- 172.21.15.3:0/1148714844 wait complete. 2024-05-21T17:42:44.147 INFO:tasks.workunit.client.0.smithi003.stderr:++ ceph osd pool ls 2024-05-21T17:42:44.147 INFO:tasks.workunit.client.0.smithi003.stderr:++ grep '^by-host-' 2024-05-21T17:42:44.196 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:44.194+0000 7f9dbba13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T17:42:44.224 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:44.222+0000 7f502964d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T17:42:44.225 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:44.224+0000 7fc84c213040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-21T17:42:44.229 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f2670b06640 1 Processor -- start 2024-05-21T17:42:44.230 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f2670b06640 1 -- start start 2024-05-21T17:42:44.231 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f2670b06640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 0x7f266c0aa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.231 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f2670b06640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f266c14b870 0x7f266c14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.231 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f2670b06640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f266c0aaa50 con 0x7f266c0a9200 2024-05-21T17:42:44.231 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f2670b06640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f266c0a6c00 con 0x7f266c14b870 2024-05-21T17:42:44.231 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f2670b06640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f266c0aad10 con 0x7f266c0a8310 2024-05-21T17:42:44.231 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f266ad9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f266c14b870 0x7f266c14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:44.231 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f2663fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 0x7f266c0aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:44.232 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f266ad9d640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f266c14b870 0x7f266c14dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37878/0 (socket says 172.21.15.3:37878) 2024-05-21T17:42:44.232 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f2663fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 0x7f266c0aa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42492/0 (socket says 172.21.15.3:42492) 2024-05-21T17:42:44.232 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.228+0000 7f266ad9d640 1 -- 172.21.15.3:0/4177076996 learned_addr learned my addr 172.21.15.3:0/4177076996 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:44.232 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.229+0000 7f266ad9d640 1 -- 172.21.15.3:0/4177076996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 msgr2=0x7f266c0aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:44.232 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.229+0000 7f266ad9d640 1 --2- 172.21.15.3:0/4177076996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 0x7f266c0aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.232 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.229+0000 7f266ad9d640 1 -- 172.21.15.3:0/4177076996 >> v1:172.21.15.203:6789/0 conn(0x7f266c0a9200 legacy=0x7f266c142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.232 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.229+0000 7f266ad9d640 1 -- 172.21.15.3:0/4177076996 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c0a64b0 con 0x7f266c14b870 2024-05-21T17:42:44.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.229+0000 7f266ad9d640 1 --2- 172.21.15.3:0/4177076996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 0x7f266c14dc60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f266400d150 tx=0x7f266402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93ddcd793e7de2d server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:44.234 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2669d9b640 1 -- 172.21.15.3:0/4177076996 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266402e9e0 con 0x7f266c14b870 2024-05-21T17:42:44.234 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2669d9b640 1 -- 172.21.15.3:0/4177076996 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266402eb80 con 0x7f266c14b870 2024-05-21T17:42:44.234 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2669d9b640 1 -- 172.21.15.3:0/4177076996 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664009600 con 0x7f266c14b870 2024-05-21T17:42:44.234 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2670b06640 1 -- 172.21.15.3:0/4177076996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 msgr2=0x7f266c14dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.234 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2670b06640 1 --2- 172.21.15.3:0/4177076996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 0x7f266c14dc60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f266400d150 tx=0x7f266402e040 comp rx=0 tx=0).stop 2024-05-21T17:42:44.234 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2670b06640 1 -- 172.21.15.3:0/4177076996 shutdown_connections 2024-05-21T17:42:44.234 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2670b06640 1 --2- 172.21.15.3:0/4177076996 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 0x7f266c14dc60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.234 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2670b06640 1 --2- 172.21.15.3:0/4177076996 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 0x7f266c0aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.234 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2670b06640 1 -- 172.21.15.3:0/4177076996 >> 172.21.15.3:0/4177076996 conn(0x7f266c075a50 msgr2=0x7f266c0a7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:44.235 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2670b06640 1 -- 172.21.15.3:0/4177076996 shutdown_connections 2024-05-21T17:42:44.235 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.230+0000 7f2670b06640 1 -- 172.21.15.3:0/4177076996 wait complete. 2024-05-21T17:42:44.235 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f2670b06640 1 Processor -- start 2024-05-21T17:42:44.235 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f2670b06640 1 -- start start 2024-05-21T17:42:44.237 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f2670b06640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 0x7f266c137a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.237 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f2670b06640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c0a9200 0x7f266c147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.237 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f2670b06640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 0x7f266c0a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.237 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f2670b06640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f266c0a6930 con 0x7f266c14b870 2024-05-21T17:42:44.237 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f2670b06640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f266c0a9b70 con 0x7f266c0a9200 2024-05-21T17:42:44.237 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f2670b06640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f266c0a95e0 con 0x7f266c0a8310 2024-05-21T17:42:44.237 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f266a59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 0x7f266c137a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:44.237 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f266ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 0x7f266c0a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:44.237 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f266ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 0x7f266c0a3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37894/0 (socket says 172.21.15.3:37894) 2024-05-21T17:42:44.237 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f266ad9d640 1 -- 172.21.15.3:0/277509770 learned_addr learned my addr 172.21.15.3:0/277509770 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f266a59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 0x7f266c137a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:42494/0 (socket says 172.21.15.3:42494) 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.232+0000 7f2663fff640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c0a9200 0x7f266c147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f266ad9d640 1 -- 172.21.15.3:0/277509770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 msgr2=0x7f266c137a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f266ad9d640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 0x7f266c137a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f266ad9d640 1 -- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c0a9200 msgr2=0x7f266c147d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f266ad9d640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c0a9200 0x7f266c147d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f266ad9d640 1 -- 172.21.15.3:0/277509770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266c148fb0 con 0x7f266c14b870 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f266ad9d640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 0x7f266c0a3d60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f266400d150 tx=0x7f2664004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f2662ffd640 1 -- 172.21.15.3:0/277509770 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664045020 con 0x7f266c14b870 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f2662ffd640 1 -- 172.21.15.3:0/277509770 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2664009e30 con 0x7f266c14b870 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f266c149270 con 0x7f266c14b870 2024-05-21T17:42:44.238 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f2662ffd640 1 -- 172.21.15.3:0/277509770 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266402e9e0 con 0x7f266c14b870 2024-05-21T17:42:44.239 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.233+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f266c147850 con 0x7f266c14b870 2024-05-21T17:42:44.239 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.234+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266c0507e0 con 0x7f266c14b870 2024-05-21T17:42:44.239 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.236+0000 7f2662ffd640 1 -- 172.21.15.3:0/277509770 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 147093+0+0 (secure 0 0 0) 0x7f2664048020 con 0x7f266c14b870 2024-05-21T17:42:44.239 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.236+0000 7f2662ffd640 20 mgrc handle_mgr_map mgrmap(e 12) 2024-05-21T17:42:44.240 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.236+0000 7f2662ffd640 4 mgrc handle_mgr_map Got map version 12 2024-05-21T17:42:44.240 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.236+0000 7f2662ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:44.240 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.236+0000 7f2662ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:44.240 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.236+0000 7f2662ffd640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f264c0af380 0x7f264c0b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.240 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.236+0000 7f266a59c640 1 -- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f264c0af380 msgr2=0x7f264c0b1840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:44.240 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.236+0000 7f2662ffd640 1 -- 172.21.15.3:0/277509770 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5566+0+0 (secure 0 0 0) 0x7f2664101c40 con 0x7f266c14b870 2024-05-21T17:42:44.240 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.236+0000 7f266a59c640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f264c0af380 0x7f264c0b1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T17:42:44.242 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.240+0000 7f2662ffd640 1 -- 172.21.15.3:0/277509770 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f266404f030 con 0x7f266c14b870 2024-05-21T17:42:44.289 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:44.286+0000 7f9dbba13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:44.313 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:44.312+0000 7f502964d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:44.314 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:44.313+0000 7fc84c213040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:44.333 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:44.331+0000 7f9dbba13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T17:42:44.358 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:44.357+0000 7f502964d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T17:42:44.358 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:44.357+0000 7fc84c213040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-21T17:42:44.433 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.431+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) -- 0x7f266c140860 con 0x7f266c14b870 2024-05-21T17:42:44.433 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.431+0000 7f2662ffd640 1 -- 172.21.15.3:0/277509770 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v24) ==== 59+0+73 (secure 0 0 0) 0x7f26640cb4a0 con 0x7f266c14b870 2024-05-21T17:42:44.439 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.437+0000 7f266a59c640 1 -- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f264c0af380 msgr2=0x7f264c0b1840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:44.439 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.437+0000 7f266a59c640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f264c0af380 0x7f264c0b1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T17:42:44.505 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:44.503+0000 7f9dbba13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T17:42:44.529 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:44.528+0000 7f502964d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T17:42:44.529 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:44.528+0000 7fc84c213040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-21T17:42:44.552 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.550+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) -- 0x7f266c0a98b0 con 0x7f266c14b870 2024-05-21T17:42:44.552 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.550+0000 7f2662ffd640 1 -- 172.21.15.3:0/277509770 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v24) ==== 59+0+73 (secure 0 0 0) 0x7f26640cada0 con 0x7f266c14b870 2024-05-21T17:42:44.552 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:44.550+0000 7f9dbba13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T17:42:44.555 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 10 mgrc shutdown 2024-05-21T17:42:44.555 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f264c0af380 msgr2=0x7f264c0b1840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T17:42:44.555 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f264c0af380 0x7f264c0b1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.555 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 msgr2=0x7f266c0a3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.555 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 0x7f266c0a3d60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f266400d150 tx=0x7f2664004a40 comp rx=0 tx=0).stop 2024-05-21T17:42:44.555 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 shutdown_connections 2024-05-21T17:42:44.555 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f264c0af380 0x7f264c0b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.555 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f266c14b870 0x7f266c0a3d60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.555 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f266c0a9200 0x7f266c147d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.555 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 1 --2- 172.21.15.3:0/277509770 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f266c0a8310 0x7f266c137a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.556 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.552+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 >> 172.21.15.3:0/277509770 conn(0x7f266c075a50 msgr2=0x7f266c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:44.556 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.553+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 shutdown_connections 2024-05-21T17:42:44.556 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.553+0000 7f2670b06640 1 -- 172.21.15.3:0/277509770 wait complete. 2024-05-21T17:42:44.564 INFO:tasks.workunit.client.0.smithi003.stderr:+ for p in `ceph osd pool ls | grep '^by-host-'` 2024-05-21T17:42:44.564 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool rm by-host-smithi003 by-host-smithi003 --yes-i-really-really-mean-it 2024-05-21T17:42:44.576 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:44.575+0000 7fc84c213040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T17:42:44.576 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:44.575+0000 7f502964d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-21T17:42:44.605 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:44.603+0000 7f9dbba13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T17:42:44.626 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:44.625+0000 7fc84c213040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T17:42:44.626 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:44.625+0000 7f502964d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-21T17:42:44.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.648+0000 7ff5f6f9d640 1 Processor -- start 2024-05-21T17:42:44.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.649+0000 7ff5f6f9d640 1 -- start start 2024-05-21T17:42:44.652 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.649+0000 7ff5f6f9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff5f006f930 0x7ff5f00aa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.652 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.649+0000 7ff5f6f9d640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f00a9480 con 0x7ff5f006e8b0 2024-05-21T17:42:44.652 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.649+0000 7ff5f6f9d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f006ecc0 con 0x7ff5f0139740 2024-05-21T17:42:44.652 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.649+0000 7ff5f6f9d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f00aa130 con 0x7ff5f006f930 2024-05-21T17:42:44.652 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.649+0000 7ff5f679c640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7ff5f006e8b0 0x7ff5f014d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:44898/0 (socket says 172.21.15.3:44898) 2024-05-21T17:42:44.652 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.649+0000 7ff5f679c640 1 -- 172.21.15.3:0/2368129992 learned_addr learned my addr 172.21.15.3:0/2368129992 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:42:44.652 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.649+0000 7ff5f5f9b640 1 --2- 172.21.15.3:0/2368129992 >> v2:172.21.15.203:3300/0 conn(0x7ff5f006f930 0x7ff5f00aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:44.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.650+0000 7ff5f4f99640 1 -- 172.21.15.3:0/2368129992 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2863273970 0 0) 0x7ff5f00a9480 con 0x7ff5f006e8b0 2024-05-21T17:42:44.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.650+0000 7ff5f4f99640 1 -- 172.21.15.3:0/2368129992 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5dc003520 con 0x7ff5f006e8b0 2024-05-21T17:42:44.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.650+0000 7ff5f4f99640 1 -- 172.21.15.3:0/2368129992 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 851598738 0 0) 0x7ff5f006ecc0 con 0x7ff5f0139740 2024-05-21T17:42:44.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.650+0000 7ff5f4f99640 1 -- 172.21.15.3:0/2368129992 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5f00a9480 con 0x7ff5f0139740 2024-05-21T17:42:44.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.650+0000 7ff5f5f9b640 1 -- 172.21.15.3:0/2368129992 >> v1:172.21.15.203:6789/0 conn(0x7ff5f0139740 legacy=0x7ff5f00a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.650+0000 7ff5f5f9b640 1 -- 172.21.15.3:0/2368129992 >> v1:172.21.15.3:6789/0 conn(0x7ff5f006e8b0 legacy=0x7ff5f014d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.650+0000 7ff5f4f99640 1 -- 172.21.15.3:0/2368129992 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3572026929 0 0) 0x7ff5dc003520 con 0x7ff5f006e8b0 2024-05-21T17:42:44.653 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.650+0000 7ff5f5f9b640 1 -- 172.21.15.3:0/2368129992 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f006ef70 con 0x7ff5f006f930 2024-05-21T17:42:44.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.651+0000 7ff5f5f9b640 1 --2- 172.21.15.3:0/2368129992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5f006f930 0x7ff5f00aa770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0097a0 tx=0x7ff5ec02b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4958291fd088536f server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:44.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.651+0000 7ff5f4f99640 1 -- 172.21.15.3:0/2368129992 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec037070 con 0x7ff5f006f930 2024-05-21T17:42:44.654 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.651+0000 7ff5f4f99640 1 -- 172.21.15.3:0/2368129992 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec030510 con 0x7ff5f006f930 2024-05-21T17:42:44.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.651+0000 7ff5f4f99640 1 -- 172.21.15.3:0/2368129992 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8001a70 con 0x7ff5f006f930 2024-05-21T17:42:44.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.652+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/2368129992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5f006f930 msgr2=0x7ff5f00aa770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.652+0000 7ff5f6f9d640 1 --2- 172.21.15.3:0/2368129992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5f006f930 0x7ff5f00aa770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0097a0 tx=0x7ff5ec02b710 comp rx=0 tx=0).stop 2024-05-21T17:42:44.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.652+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/2368129992 shutdown_connections 2024-05-21T17:42:44.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.652+0000 7ff5f6f9d640 1 --2- 172.21.15.3:0/2368129992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5f006f930 0x7ff5f00aa770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.652+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/2368129992 >> 172.21.15.3:0/2368129992 conn(0x7ff5f00a7820 msgr2=0x7ff5f00a7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:44.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.652+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/2368129992 shutdown_connections 2024-05-21T17:42:44.655 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.652+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/2368129992 wait complete. 2024-05-21T17:42:44.657 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.653+0000 7ff5f6f9d640 1 Processor -- start 2024-05-21T17:42:44.657 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.653+0000 7ff5f6f9d640 1 -- start start 2024-05-21T17:42:44.657 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.653+0000 7ff5f6f9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5f006e8b0 0x7ff5f01429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.653+0000 7ff5f6f9d640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff5f006f930 0x7ff5f00b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.653+0000 7ff5f6f9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5f0139740 0x7ff5f00b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.653+0000 7ff5f6f9d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f00a5620 con 0x7ff5f006e8b0 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.653+0000 7ff5f6f9d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f006f230 con 0x7ff5f0139740 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.653+0000 7ff5f6f9d640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f00a7460 con 0x7ff5f006f930 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.653+0000 7ff5f5f9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5f006e8b0 0x7ff5f01429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f5f9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5f006e8b0 0x7ff5f01429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37906/0 (socket says 172.21.15.3:37906) 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f5f9b640 1 -- 172.21.15.3:0/581542072 learned_addr learned my addr 172.21.15.3:0/581542072 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f579a640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff5f006f930 0x7ff5f00b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f679c640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5f0139740 0x7ff5f00b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f5f9b640 1 -- 172.21.15.3:0/581542072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff5f006f930 msgr2=0x7ff5f00b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.658 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f5f9b640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff5f006f930 0x7ff5f00b72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f5f9b640 1 -- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5f0139740 msgr2=0x7ff5f00b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:44.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f5f9b640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5f0139740 0x7ff5f00b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:44.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f5f9b640 1 -- 172.21.15.3:0/581542072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f013e450 con 0x7ff5f006e8b0 2024-05-21T17:42:44.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f5f9b640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5f006e8b0 0x7ff5f01429a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec000c00 tx=0x7ff5ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:44.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5e67fc640 1 -- 172.21.15.3:0/581542072 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec040020 con 0x7ff5f006e8b0 2024-05-21T17:42:44.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5e67fc640 1 -- 172.21.15.3:0/581542072 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec038040 con 0x7ff5f006e8b0 2024-05-21T17:42:44.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f013e710 con 0x7ff5f006e8b0 2024-05-21T17:42:44.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5e67fc640 1 -- 172.21.15.3:0/581542072 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec037070 con 0x7ff5f006e8b0 2024-05-21T17:42:44.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.654+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f0141ca0 con 0x7ff5f006e8b0 2024-05-21T17:42:44.659 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.655+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f00507e0 con 0x7ff5f006e8b0 2024-05-21T17:42:44.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.657+0000 7ff5e67fc640 1 -- 172.21.15.3:0/581542072 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 147093+0+0 (secure 0 0 0) 0x7ff5ec043020 con 0x7ff5f006e8b0 2024-05-21T17:42:44.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.657+0000 7ff5e67fc640 20 mgrc handle_mgr_map mgrmap(e 12) 2024-05-21T17:42:44.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.657+0000 7ff5e67fc640 4 mgrc handle_mgr_map Got map version 12 2024-05-21T17:42:44.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.657+0000 7ff5e67fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:44.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.657+0000 7ff5e67fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:44.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.657+0000 7ff5e67fc640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7ff5d40af300 0x7ff5d40b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:44.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.657+0000 7ff5f579a640 1 -- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7ff5d40af300 msgr2=0x7ff5d40b17c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:44.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.657+0000 7ff5e67fc640 1 -- 172.21.15.3:0/581542072 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5566+0+0 (secure 0 0 0) 0x7ff5ec0fea50 con 0x7ff5f006e8b0 2024-05-21T17:42:44.660 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.657+0000 7ff5f579a640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7ff5d40af300 0x7ff5d40b17c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T17:42:44.666 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.664+0000 7ff5e67fc640 1 -- 172.21.15.3:0/581542072 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff5ec04a030 con 0x7ff5f006e8b0 2024-05-21T17:42:44.730 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:44.728+0000 7f9dbba13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:44.740 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:44.739+0000 7fc84c213040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:44.741 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:44.740+0000 7f502964d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-21T17:42:44.858 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.856+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "by-host-smithi003", "pool2": "by-host-smithi003", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff5f01459c0 con 0x7ff5f006e8b0 2024-05-21T17:42:44.861 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.858+0000 7ff5f579a640 1 -- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7ff5d40af300 msgr2=0x7ff5d40b17c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:44.862 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:44.858+0000 7ff5f579a640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7ff5d40af300 0x7ff5d40b17c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T17:42:45.069 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:45.067+0000 7f9dbba13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T17:42:45.073 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:45.072+0000 7fc84c213040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T17:42:45.073 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:45.072+0000 7f502964d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-21T17:42:45.090 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.088+0000 7ff5e67fc640 1 -- 172.21.15.3:0/581542072 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "by-host-smithi003", "pool2": "by-host-smithi003", "yes_i_really_really_mean_it": true}]=0 pool 'by-host-smithi003' removed v25) ==== 187+0+0 (secure 0 0 0) 0x7ff5ec0c8110 con 0x7ff5f006e8b0 2024-05-21T17:42:45.219 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:45.217+0000 7f9dbba13040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T17:42:45.229 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.228+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "by-host-smithi003", "pool2": "by-host-smithi003", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff5f006ddd0 con 0x7ff5f006e8b0 2024-05-21T17:42:45.230 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.228+0000 7ff5e67fc640 1 -- 172.21.15.3:0/581542072 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "by-host-smithi003", "pool2": "by-host-smithi003", "yes_i_really_really_mean_it": true}]=0 pool 'by-host-smithi003' does not exist v25) ==== 194+0+0 (secure 0 0 0) 0x7ff5ec0c7a10 con 0x7ff5f006e8b0 2024-05-21T17:42:45.230 INFO:tasks.workunit.client.0.smithi003.stderr:pool 'by-host-smithi003' does not exist 2024-05-21T17:42:45.232 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 10 mgrc shutdown 2024-05-21T17:42:45.232 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7ff5d40af300 msgr2=0x7ff5d40b17c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T17:42:45.232 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7ff5d40af300 0x7ff5d40b17c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5f006e8b0 msgr2=0x7ff5f01429a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:45.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5f006e8b0 0x7ff5f01429a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec000c00 tx=0x7ff5ec0046c0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 shutdown_connections 2024-05-21T17:42:45.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7ff5d40af300 0x7ff5d40b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5f0139740 0x7ff5f00b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff5f006f930 0x7ff5f00b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 --2- 172.21.15.3:0/581542072 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff5f006e8b0 0x7ff5f01429a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 >> 172.21.15.3:0/581542072 conn(0x7ff5f00a7820 msgr2=0x7ff5f01439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:45.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 shutdown_connections 2024-05-21T17:42:45.233 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.230+0000 7ff5f6f9d640 1 -- 172.21.15.3:0/581542072 wait complete. 2024-05-21T17:42:45.241 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:45.240+0000 7f502964d040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T17:42:45.241 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:45.240+0000 7fc84c213040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-21T17:42:45.242 INFO:tasks.workunit.client.0.smithi003.stderr:+ for p in `ceph osd pool ls | grep '^by-host-'` 2024-05-21T17:42:45.242 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph osd pool rm by-host-smithi203 by-host-smithi203 --yes-i-really-really-mean-it 2024-05-21T17:42:45.327 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.325+0000 7f9829c83640 1 Processor -- start 2024-05-21T17:42:45.327 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.325+0000 7f9829c83640 1 -- start start 2024-05-21T17:42:45.329 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9829c83640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f98240aa1f0 0x7f98240a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:45.329 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9829c83640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f98240a9050 0x7f9824142220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:45.329 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9829c83640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98240a67c0 con 0x7f98240aa5d0 2024-05-21T17:42:45.329 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9829c83640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7f98240a6a80 con 0x7f98240a9050 2024-05-21T17:42:45.329 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9829c83640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f98240a7890 con 0x7f98240aa1f0 2024-05-21T17:42:45.329 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9823fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f98240a9050 0x7f9824142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:45.329 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9823fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7f98240a9050 0x7f9824142220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40616/0 (socket says 172.21.15.3:40616) 2024-05-21T17:42:45.329 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9823fff640 1 -- 172.21.15.3:0/2774691507 learned_addr learned my addr 172.21.15.3:0/2774691507 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:45.329 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f98237fe640 1 --2- 172.21.15.3:0/2774691507 >> v2:172.21.15.203:3300/0 conn(0x7f98240aa1f0 0x7f98240a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:45.329 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9823fff640 1 -- 172.21.15.3:0/2774691507 >> v2:172.21.15.203:3300/0 conn(0x7f98240aa1f0 msgr2=0x7f98240a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:45.330 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9823fff640 1 --2- 172.21.15.3:0/2774691507 >> v2:172.21.15.203:3300/0 conn(0x7f98240aa1f0 0x7f98240a6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.330 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9823fff640 1 -- 172.21.15.3:0/2774691507 >> v1:172.21.15.203:6789/0 conn(0x7f98240aa5d0 legacy=0x7f9824143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:45.330 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.326+0000 7f9823fff640 1 -- 172.21.15.3:0/2774691507 --> v2:172.21.15.3:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98240a7530 con 0x7f98240a9050 2024-05-21T17:42:45.331 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.327+0000 7f9823fff640 1 --2- 172.21.15.3:0/2774691507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240a9050 0x7f9824142220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98240a5e60 tx=0x7f981c02e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcf6faa241ca225 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:45.331 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.327+0000 7f98227fc640 1 -- 172.21.15.3:0/2774691507 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c02ea90 con 0x7f98240a9050 2024-05-21T17:42:45.331 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.327+0000 7f98227fc640 1 -- 172.21.15.3:0/2774691507 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c02ec30 con 0x7f98240a9050 2024-05-21T17:42:45.331 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.327+0000 7f98227fc640 1 -- 172.21.15.3:0/2774691507 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c009600 con 0x7f98240a9050 2024-05-21T17:42:45.331 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.328+0000 7f9829c83640 1 -- 172.21.15.3:0/2774691507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240a9050 msgr2=0x7f9824142220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:45.331 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.328+0000 7f9829c83640 1 --2- 172.21.15.3:0/2774691507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240a9050 0x7f9824142220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98240a5e60 tx=0x7f981c02e0f0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.332 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.328+0000 7f9829c83640 1 -- 172.21.15.3:0/2774691507 shutdown_connections 2024-05-21T17:42:45.332 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.328+0000 7f9829c83640 1 --2- 172.21.15.3:0/2774691507 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240a9050 0x7f9824142220 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.332 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.328+0000 7f9829c83640 1 --2- 172.21.15.3:0/2774691507 >> v2:172.21.15.203:3300/0 conn(0x7f98240aa1f0 0x7f98240a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.332 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.328+0000 7f9829c83640 1 -- 172.21.15.3:0/2774691507 >> 172.21.15.3:0/2774691507 conn(0x7f9824075c70 msgr2=0x7f98240a7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:45.332 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.328+0000 7f9829c83640 1 -- 172.21.15.3:0/2774691507 shutdown_connections 2024-05-21T17:42:45.332 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.328+0000 7f9829c83640 1 -- 172.21.15.3:0/2774691507 wait complete. 2024-05-21T17:42:45.332 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.329+0000 7f9829c83640 1 Processor -- start 2024-05-21T17:42:45.332 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.329+0000 7f9829c83640 1 -- start start 2024-05-21T17:42:45.334 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9829c83640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98240a9050 0x7f98240a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:45.334 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9829c83640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98240aa1f0 0x7f98240a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:45.334 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9829c83640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240aa5d0 0x7f98240a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:45.334 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9829c83640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9824143dc0 con 0x7f98240aa5d0 2024-05-21T17:42:45.334 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9829c83640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9824143c40 con 0x7f98240a9050 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9829c83640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f98240aae30 con 0x7f98240aa1f0 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9823fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240aa5d0 0x7f98240a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9823fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240aa5d0 0x7f98240a3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40618/0 (socket says 172.21.15.3:40618) 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9823fff640 1 -- 172.21.15.3:0/1347216684 learned_addr learned my addr 172.21.15.3:0/1347216684 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9822ffd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98240aa1f0 0x7f98240a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f98237fe640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98240a9050 0x7f98240a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9823fff640 1 -- 172.21.15.3:0/1347216684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98240aa1f0 msgr2=0x7f98240a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9823fff640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98240aa1f0 0x7f98240a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9823fff640 1 -- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98240a9050 msgr2=0x7f98240a29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9823fff640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98240a9050 0x7f98240a29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9823fff640 1 -- 172.21.15.3:0/1347216684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98240a42e0 con 0x7f98240aa5d0 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f98237fe640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98240a9050 0x7f98240a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:45.335 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.330+0000 7f9822ffd640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98240aa1f0 0x7f98240a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:45.336 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.331+0000 7f9823fff640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240aa5d0 0x7f98240a3ae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f981c00d240 tx=0x7f981c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:45.336 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.331+0000 7f980bfff640 1 -- 172.21.15.3:0/1347216684 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c02ea90 con 0x7f98240aa5d0 2024-05-21T17:42:45.336 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.331+0000 7f980bfff640 1 -- 172.21.15.3:0/1347216684 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981c045070 con 0x7f98240aa5d0 2024-05-21T17:42:45.336 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.331+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98240a45a0 con 0x7f98240aa5d0 2024-05-21T17:42:45.336 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.331+0000 7f980bfff640 1 -- 172.21.15.3:0/1347216684 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c00b720 con 0x7f98240aa5d0 2024-05-21T17:42:45.336 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.331+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982409efa0 con 0x7f98240aa5d0 2024-05-21T17:42:45.336 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.332+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9824050850 con 0x7f98240aa5d0 2024-05-21T17:42:45.337 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.333+0000 7f980bfff640 1 -- 172.21.15.3:0/1347216684 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 147093+0+0 (secure 0 0 0) 0x7f981c036a60 con 0x7f98240aa5d0 2024-05-21T17:42:45.337 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.333+0000 7f980bfff640 20 mgrc handle_mgr_map mgrmap(e 12) 2024-05-21T17:42:45.337 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.334+0000 7f980bfff640 4 mgrc handle_mgr_map Got map version 12 2024-05-21T17:42:45.337 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.334+0000 7f980bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:45.337 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.334+0000 7f980bfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:45.337 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.334+0000 7f980bfff640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f98100af380 0x7f98100b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:45.337 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.334+0000 7f98237fe640 1 -- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f98100af380 msgr2=0x7f98100b1840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:45.337 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.334+0000 7f980bfff640 1 -- 172.21.15.3:0/1347216684 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5210+0+0 (secure 0 0 0) 0x7f981c0ffb00 con 0x7f98240aa5d0 2024-05-21T17:42:45.338 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.334+0000 7f98237fe640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f98100af380 0x7f98100b1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T17:42:45.340 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.338+0000 7f980bfff640 1 -- 172.21.15.3:0/1347216684 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f981c0c9430 con 0x7f98240aa5d0 2024-05-21T17:42:45.458 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:45.456+0000 7f9dbba13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T17:42:45.458 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:45.457+0000 7fc84c213040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T17:42:45.459 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:45.458+0000 7f502964d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-21T17:42:45.510 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:45.508+0000 7fc84c213040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T17:42:45.510 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:45.509+0000 7f502964d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T17:42:45.510 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:45.508+0000 7f9dbba13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-21T17:42:45.537 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.535+0000 7f98237fe640 1 -- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f98100af380 msgr2=0x7f98100b1840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:45.537 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.535+0000 7f98237fe640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f98100af380 0x7f98100b1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T17:42:45.547 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.545+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "by-host-smithi203", "pool2": "by-host-smithi203", "yes_i_really_really_mean_it": true} v 0) -- 0x7f98240b8540 con 0x7f98240aa5d0 2024-05-21T17:42:45.627 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:45.625+0000 7fc84c213040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T17:42:45.627 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:45.626+0000 7f502964d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T17:42:45.630 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:45.628+0000 7f9dbba13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-21T17:42:45.899 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:45.898+0000 7fc84c213040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T17:42:45.900 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:45.899+0000 7f502964d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T17:42:45.912 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:45.910+0000 7f9dbba13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-21T17:42:45.938 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.936+0000 7f98237fe640 1 -- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f98100af380 msgr2=0x7f98100b1840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:45.938 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:45.936+0000 7f98237fe640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f98100af380 0x7f98100b1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-21T17:42:45.944 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:45.943+0000 7fc84c213040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T17:42:45.945 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:45.944+0000 7f502964d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T17:42:45.957 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:45.956+0000 7f9dbba13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-21T17:42:46.091 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.089+0000 7f980bfff640 1 -- 172.21.15.3:0/1347216684 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "by-host-smithi203", "pool2": "by-host-smithi203", "yes_i_really_really_mean_it": true}]=0 pool 'by-host-smithi203' removed v26) ==== 187+0+0 (secure 0 0 0) 0x7f981c0c8d30 con 0x7f98240aa5d0 2024-05-21T17:42:46.092 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.091+0000 7fc84c213040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T17:42:46.094 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:46.093+0000 7f502964d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T17:42:46.110 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:46.108+0000 7f9dbba13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-21T17:42:46.207 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.205+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "by-host-smithi203", "pool2": "by-host-smithi203", "yes_i_really_really_mean_it": true} v 0) -- 0x7f98240a6130 con 0x7f98240aa5d0 2024-05-21T17:42:46.208 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.206+0000 7f980bfff640 1 -- 172.21.15.3:0/1347216684 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "by-host-smithi203", "pool2": "by-host-smithi203", "yes_i_really_really_mean_it": true}]=0 pool 'by-host-smithi203' does not exist v26) ==== 194+0+0 (secure 0 0 0) 0x7f981c0cd0e0 con 0x7f98240aa5d0 2024-05-21T17:42:46.208 INFO:tasks.workunit.client.0.smithi003.stderr:pool 'by-host-smithi203' does not exist 2024-05-21T17:42:46.210 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 10 mgrc shutdown 2024-05-21T17:42:46.210 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f98100af380 msgr2=0x7f98100b1840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f98100af380 0x7f98100b1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240aa5d0 msgr2=0x7f98240a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240aa5d0 0x7f98240a3ae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f981c00d240 tx=0x7f981c004a40 comp rx=0 tx=0).stop 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 shutdown_connections 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f98100af380 0x7f98100b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f98240aa5d0 0x7f98240a3ae0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f98240aa1f0 0x7f98240a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 --2- 172.21.15.3:0/1347216684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98240a9050 0x7f98240a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 >> 172.21.15.3:0/1347216684 conn(0x7f9824075c70 msgr2=0x7f9824068660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 shutdown_connections 2024-05-21T17:42:46.211 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.208+0000 7f9829c83640 1 -- 172.21.15.3:0/1347216684 wait complete. 2024-05-21T17:42:46.219 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph config rm mgr mgr/localpool/subtree 2024-05-21T17:42:46.239 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.238+0000 7fc84c213040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T17:42:46.240 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:46.239+0000 7f502964d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T17:42:46.253 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:46.251+0000 7f9dbba13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-21T17:42:46.289 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.288+0000 7fc84c213040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T17:42:46.290 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:46.289+0000 7f502964d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T17:42:46.303 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.301+0000 7f4deb4ed640 1 Processor -- start 2024-05-21T17:42:46.303 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.301+0000 7f4deb4ed640 1 -- start start 2024-05-21T17:42:46.305 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4deb4ed640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4de40a84d0 0x7f4de4072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.305 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4deb4ed640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de4147800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.305 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4deb4ed640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4de40a7150 con 0x7f4de4147d40 2024-05-21T17:42:46.305 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4deb4ed640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f4de40a88b0 con 0x7f4de40a84d0 2024-05-21T17:42:46.305 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4deb4ed640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f4de4148490 con 0x7f4de4072780 2024-05-21T17:42:46.305 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4de8a88640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de4147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:46.305 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4de9a8a640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7f4de4147d40 0x7f4de406c3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38466/0 (socket says 172.21.15.3:38466) 2024-05-21T17:42:46.305 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4de9a8a640 1 -- 172.21.15.3:0/4027624793 learned_addr learned my addr 172.21.15.3:0/4027624793 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:42:46.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4de9289640 1 --2- 172.21.15.3:0/4027624793 >> v2:172.21.15.203:3300/0 conn(0x7f4de40a84d0 0x7f4de4072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:46.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4dd7fff640 1 -- 172.21.15.3:0/4027624793 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285974610 0 0) 0x7f4de40a7150 con 0x7f4de4147d40 2024-05-21T17:42:46.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4de8a88640 1 -- 172.21.15.3:0/4027624793 >> v2:172.21.15.203:3300/0 conn(0x7f4de40a84d0 msgr2=0x7f4de4072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:46.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.302+0000 7f4de8a88640 1 --2- 172.21.15.3:0/4027624793 >> v2:172.21.15.203:3300/0 conn(0x7f4de40a84d0 0x7f4de4072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.303+0000 7f4de8a88640 1 -- 172.21.15.3:0/4027624793 >> v1:172.21.15.3:6789/0 conn(0x7f4de4147d40 legacy=0x7f4de406c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:46.306 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.303+0000 7f4de8a88640 1 -- 172.21.15.3:0/4027624793 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de41481b0 con 0x7f4de4072780 2024-05-21T17:42:46.306 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:46.303+0000 7f9dbba13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-21T17:42:46.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.303+0000 7f4de8a88640 1 --2- 172.21.15.3:0/4027624793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de4147800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009d90 tx=0x7f4dd802beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdbdca2d77548978 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:46.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.303+0000 7f4dd7fff640 1 -- 172.21.15.3:0/4027624793 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8039070 con 0x7f4de4072780 2024-05-21T17:42:46.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.303+0000 7f4dd7fff640 1 -- 172.21.15.3:0/4027624793 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd802cc60 con 0x7f4de4072780 2024-05-21T17:42:46.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.304+0000 7f4dd7fff640 1 -- 172.21.15.3:0/4027624793 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8034660 con 0x7f4de4072780 2024-05-21T17:42:46.309 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.304+0000 7f4deb4ed640 1 -- 172.21.15.3:0/4027624793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 msgr2=0x7f4de4147800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:46.310 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.304+0000 7f4deb4ed640 1 --2- 172.21.15.3:0/4027624793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de4147800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009d90 tx=0x7f4dd802beb0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.310 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.304+0000 7f4deb4ed640 1 -- 172.21.15.3:0/4027624793 shutdown_connections 2024-05-21T17:42:46.310 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.304+0000 7f4deb4ed640 1 --2- 172.21.15.3:0/4027624793 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de4147800 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.310 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.304+0000 7f4deb4ed640 1 --2- 172.21.15.3:0/4027624793 >> v2:172.21.15.203:3300/0 conn(0x7f4de40a84d0 0x7f4de4072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.310 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.304+0000 7f4deb4ed640 1 -- 172.21.15.3:0/4027624793 >> 172.21.15.3:0/4027624793 conn(0x7f4de4075a50 msgr2=0x7f4de40a7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:46.310 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.304+0000 7f4deb4ed640 1 -- 172.21.15.3:0/4027624793 shutdown_connections 2024-05-21T17:42:46.310 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.304+0000 7f4deb4ed640 1 -- 172.21.15.3:0/4027624793 wait complete. 2024-05-21T17:42:46.310 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.305+0000 7f4deb4ed640 1 Processor -- start 2024-05-21T17:42:46.310 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.305+0000 7f4deb4ed640 1 -- start start 2024-05-21T17:42:46.310 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4deb4ed640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de40b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4deb4ed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4de40a84d0 0x7f4de41cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4deb4ed640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4de4147d40 0x7f4de40a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4deb4ed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4de406cc80 con 0x7f4de40a84d0 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4deb4ed640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4de406cb00 con 0x7f4de4147d40 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4deb4ed640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f4de406ce00 con 0x7f4de4072780 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de9289640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de40b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de8a88640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4de40a84d0 0x7f4de41cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de9289640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de40b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51576/0 (socket says 172.21.15.3:51576) 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de9289640 1 -- 172.21.15.3:0/131832249 learned_addr learned my addr 172.21.15.3:0/131832249 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de9a8a640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4de4147d40 0x7f4de40a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de8a88640 1 -- 172.21.15.3:0/131832249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 msgr2=0x7f4de40b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de8a88640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de40b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.311 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de8a88640 1 -- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4de4147d40 msgr2=0x7f4de40a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:46.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de8a88640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4de4147d40 0x7f4de40a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de8a88640 1 -- 172.21.15.3:0/131832249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de40b51a0 con 0x7f4de40a84d0 2024-05-21T17:42:46.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.306+0000 7f4de9289640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de40b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:46.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.307+0000 7f4de8a88640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4de40a84d0 0x7f4de41cdc50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8033000 tx=0x7f4dd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:46.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.307+0000 7f4dd5ffb640 1 -- 172.21.15.3:0/131832249 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8039050 con 0x7f4de40a84d0 2024-05-21T17:42:46.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.307+0000 7f4dd5ffb640 1 -- 172.21.15.3:0/131832249 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd8045070 con 0x7f4de40a84d0 2024-05-21T17:42:46.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.307+0000 7f4dd5ffb640 1 -- 172.21.15.3:0/131832249 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd803e870 con 0x7f4de40a84d0 2024-05-21T17:42:46.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.307+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de40b5460 con 0x7f4de40a84d0 2024-05-21T17:42:46.312 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.307+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de40b5bb0 con 0x7f4de40a84d0 2024-05-21T17:42:46.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.308+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de4069f80 con 0x7f4de40a84d0 2024-05-21T17:42:46.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.309+0000 7f4dd5ffb640 1 -- 172.21.15.3:0/131832249 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 147093+0+0 (secure 0 0 0) 0x7f4dd803ea10 con 0x7f4de40a84d0 2024-05-21T17:42:46.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.309+0000 7f4dd5ffb640 20 mgrc handle_mgr_map mgrmap(e 12) 2024-05-21T17:42:46.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.310+0000 7f4dd5ffb640 4 mgrc handle_mgr_map Got map version 12 2024-05-21T17:42:46.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.310+0000 7f4dd5ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:46.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.310+0000 7f4dd5ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:46.313 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.310+0000 7f4dd5ffb640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f4dc80af300 0x7f4dc80b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.314 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.311+0000 7f4de9289640 1 -- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f4dc80af300 msgr2=0x7f4dc80b17c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:46.314 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.311+0000 7f4dd5ffb640 1 -- 172.21.15.3:0/131832249 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4854+0+0 (secure 0 0 0) 0x7f4dd80ff930 con 0x7f4de40a84d0 2024-05-21T17:42:46.314 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.311+0000 7f4de9289640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f4dc80af300 0x7f4dc80b17c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T17:42:46.316 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.314+0000 7f4dd5ffb640 1 -- 172.21.15.3:0/131832249 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4dd80c9320 con 0x7f4de40a84d0 2024-05-21T17:42:46.345 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.344+0000 7fc84c213040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:46.345 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:46.344+0000 7f502964d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:46.379 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:46.377+0000 7f9dbba13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-21T17:42:46.389 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.388+0000 7fc84c213040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T17:42:46.390 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:46.389+0000 7f502964d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T17:42:46.424 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:46.423+0000 7f9dbba13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-21T17:42:46.514 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.512+0000 7f4de9289640 1 -- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f4dc80af300 msgr2=0x7f4dc80b17c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:46.514 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.512+0000 7f4de9289640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f4dc80af300 0x7f4dc80b17c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-21T17:42:46.528 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.526+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/subtree"} v 0) -- 0x7f4de40b80f0 con 0x7f4de40a84d0 2024-05-21T17:42:46.534 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.532+0000 7f4dd5ffb640 1 -- 172.21.15.3:0/131832249 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/subtree"}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7f4dd80c8c20 con 0x7f4de40a84d0 2024-05-21T17:42:46.647 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.645+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/subtree"} v 0) -- 0x7f4de406f280 con 0x7f4de40a84d0 2024-05-21T17:42:46.648 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.646+0000 7f4dd5ffb640 1 -- 172.21.15.3:0/131832249 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/subtree"}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7f4dd80ccfd0 con 0x7f4de40a84d0 2024-05-21T17:42:46.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 10 mgrc shutdown 2024-05-21T17:42:46.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f4dc80af300 msgr2=0x7f4dc80b17c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T17:42:46.650 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f4dc80af300 0x7f4dc80b17c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4de40a84d0 msgr2=0x7f4de41cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:46.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4de40a84d0 0x7f4de41cdc50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8033000 tx=0x7f4dd8004a40 comp rx=0 tx=0).stop 2024-05-21T17:42:46.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 shutdown_connections 2024-05-21T17:42:46.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7f4dc80af300 0x7f4dc80b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4de4147d40 0x7f4de40a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4de40a84d0 0x7f4de41cdc50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 --2- 172.21.15.3:0/131832249 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f4de4072780 0x7f4de40b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 >> 172.21.15.3:0/131832249 conn(0x7f4de4075a50 msgr2=0x7f4de4073190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:46.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 shutdown_connections 2024-05-21T17:42:46.651 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.648+0000 7f4deb4ed640 1 -- 172.21.15.3:0/131832249 wait complete. 2024-05-21T17:42:46.659 INFO:tasks.workunit.client.0.smithi003.stderr:+ ceph config rm mgr mgr/localpool/failure_domain 2024-05-21T17:42:46.744 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.741+0000 7fec94fe8640 1 Processor -- start 2024-05-21T17:42:46.744 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.741+0000 7fec94fe8640 1 -- start start 2024-05-21T17:42:46.744 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.741+0000 7fec94fe8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.744 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.741+0000 7fec94fe8640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec9014a140 0x7fec900a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.744 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.741+0000 7fec94fe8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec900a71f0 con 0x7fec9014a520 2024-05-21T17:42:46.744 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.741+0000 7fec94fe8640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7fec900a88b0 con 0x7fec9014a140 2024-05-21T17:42:46.744 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.741+0000 7fec94fe8640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fec90149840 con 0x7fec900a84d0 2024-05-21T17:42:46.744 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.742+0000 7fec8e59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:46.745 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.742+0000 7fec87fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7fec9014a140 0x7fec900a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:46.745 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.742+0000 7fec8e59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51582/0 (socket says 172.21.15.3:51582) 2024-05-21T17:42:46.745 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.742+0000 7fec8e59c640 1 -- 172.21.15.3:0/3012303348 learned_addr learned my addr 172.21.15.3:0/3012303348 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:46.745 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.742+0000 7fec8e59c640 1 -- 172.21.15.3:0/3012303348 >> v2:172.21.15.3:3300/0 conn(0x7fec9014a140 msgr2=0x7fec900a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:46.745 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.742+0000 7fec8e59c640 1 --2- 172.21.15.3:0/3012303348 >> v2:172.21.15.3:3300/0 conn(0x7fec9014a140 0x7fec900a96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.745 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.742+0000 7fec8e59c640 1 -- 172.21.15.3:0/3012303348 >> v1:172.21.15.203:6789/0 conn(0x7fec9014a520 legacy=0x7fec9006c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:46.745 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.742+0000 7fec8e59c640 1 -- 172.21.15.3:0/3012303348 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec90149a70 con 0x7fec900a84d0 2024-05-21T17:42:46.745 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.742+0000 7fec87fff640 1 --2- 172.21.15.3:0/3012303348 >> v2:172.21.15.3:3300/0 conn(0x7fec9014a140 0x7fec900a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:46.746 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.743+0000 7fec8e59c640 1 --2- 172.21.15.3:0/3012303348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a6cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec80009870 tx=0x7fec8002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd13bd8f01d2107 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:46.746 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.743+0000 7fec8dd9b640 1 -- 172.21.15.3:0/3012303348 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80035020 con 0x7fec900a84d0 2024-05-21T17:42:46.747 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.743+0000 7fec8dd9b640 1 -- 172.21.15.3:0/3012303348 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec80031640 con 0x7fec900a84d0 2024-05-21T17:42:46.747 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.743+0000 7fec8dd9b640 1 -- 172.21.15.3:0/3012303348 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8002d040 con 0x7fec900a84d0 2024-05-21T17:42:46.747 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.744+0000 7fec94fe8640 1 -- 172.21.15.3:0/3012303348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 msgr2=0x7fec900a6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:46.747 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.744+0000 7fec94fe8640 1 --2- 172.21.15.3:0/3012303348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a6cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec80009870 tx=0x7fec8002b8f0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.747 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.744+0000 7fec94fe8640 1 -- 172.21.15.3:0/3012303348 shutdown_connections 2024-05-21T17:42:46.747 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.744+0000 7fec94fe8640 1 --2- 172.21.15.3:0/3012303348 >> v2:172.21.15.3:3300/0 conn(0x7fec9014a140 0x7fec900a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.747 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.744+0000 7fec94fe8640 1 --2- 172.21.15.3:0/3012303348 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a6cb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.747 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.744+0000 7fec94fe8640 1 -- 172.21.15.3:0/3012303348 >> 172.21.15.3:0/3012303348 conn(0x7fec90075a50 msgr2=0x7fec900a7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:46.747 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.744+0000 7fec94fe8640 1 -- 172.21.15.3:0/3012303348 shutdown_connections 2024-05-21T17:42:46.747 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.744+0000 7fec94fe8640 1 -- 172.21.15.3:0/3012303348 wait complete. 2024-05-21T17:42:46.749 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.745+0000 7fec94fe8640 1 Processor -- start 2024-05-21T17:42:46.749 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.745+0000 7fec94fe8640 1 -- start start 2024-05-21T17:42:46.749 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec94fe8640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.749 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec94fe8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec9014a140 0x7fec9009ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.749 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec94fe8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec9014a520 0x7fec9009f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.749 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec94fe8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec9006efb0 con 0x7fec9014a520 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec94fe8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fec90149d10 con 0x7fec9014a140 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec94fe8640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fec9006fb30 con 0x7fec900a84d0 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec8e59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec8ed9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec9014a520 0x7fec9009f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec8e59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51596/0 (socket says 172.21.15.3:51596) 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec8e59c640 1 -- 172.21.15.3:0/3179644757 learned_addr learned my addr 172.21.15.3:0/3179644757 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec87fff640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec9014a140 0x7fec9009ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec8e59c640 1 -- 172.21.15.3:0/3179644757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec9014a140 msgr2=0x7fec9009ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec8e59c640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec9014a140 0x7fec9009ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec8e59c640 1 -- 172.21.15.3:0/3179644757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec9014a520 msgr2=0x7fec9009f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec8e59c640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec9014a520 0x7fec9009f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec8e59c640 1 -- 172.21.15.3:0/3179644757 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9009fba0 con 0x7fec900a84d0 2024-05-21T17:42:46.750 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec8e59c640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a4060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec8002b930 tx=0x7fec800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:46.751 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec86ffd640 1 -- 172.21.15.3:0/3179644757 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80040070 con 0x7fec900a84d0 2024-05-21T17:42:46.751 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec86ffd640 1 -- 172.21.15.3:0/3179644757 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8003e070 con 0x7fec900a84d0 2024-05-21T17:42:46.751 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec94fe8640 1 -- 172.21.15.3:0/3179644757 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec900a4c70 con 0x7fec900a84d0 2024-05-21T17:42:46.751 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec86ffd640 1 -- 172.21.15.3:0/3179644757 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80035040 con 0x7fec900a84d0 2024-05-21T17:42:46.751 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.746+0000 7fec94fe8640 1 -- 172.21.15.3:0/3179644757 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fec901d0680 con 0x7fec900a84d0 2024-05-21T17:42:46.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.748+0000 7fec94fe8640 1 -- 172.21.15.3:0/3179644757 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec90069f80 con 0x7fec900a84d0 2024-05-21T17:42:46.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.749+0000 7fec86ffd640 1 -- 172.21.15.3:0/3179644757 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 12) ==== 147093+0+0 (secure 0 0 0) 0x7fec80038020 con 0x7fec900a84d0 2024-05-21T17:42:46.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.749+0000 7fec86ffd640 20 mgrc handle_mgr_map mgrmap(e 12) 2024-05-21T17:42:46.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.750+0000 7fec86ffd640 4 mgrc handle_mgr_map Got map version 12 2024-05-21T17:42:46.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.750+0000 7fec86ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:46.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.750+0000 7fec86ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] 2024-05-21T17:42:46.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.750+0000 7fec86ffd640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fec700af380 0x7fec700b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:46.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.750+0000 7fec87fff640 1 -- 172.21.15.3:0/3179644757 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fec700af380 msgr2=0x7fec700b1840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:46.752 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.750+0000 7fec86ffd640 1 -- 172.21.15.3:0/3179644757 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 4854+0+0 (secure 0 0 0) 0x7fec800fe5e0 con 0x7fec900a84d0 2024-05-21T17:42:46.753 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.750+0000 7fec87fff640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fec700af380 0x7fec700b1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-21T17:42:46.756 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.754+0000 7fec86ffd640 1 -- 172.21.15.3:0/3179644757 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fec901d0680 con 0x7fec900a84d0 2024-05-21T17:42:46.813 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.812+0000 7fc84c213040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T17:42:46.816 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:46.815+0000 7f502964d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T17:42:46.904 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:46.902+0000 7f9dbba13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-21T17:42:46.932 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.931+0000 7fc84c213040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T17:42:46.935 INFO:tasks.ceph.mgr.y.smithi203.stderr:2024-05-21T17:42:46.934+0000 7f502964d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T17:42:46.949 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.948+0000 7fec86ffd640 1 -- 172.21.15.3:0/3179644757 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mgrmap(e 13) ==== 146702+0+0 (secure 0 0 0) 0x7fec800fce50 con 0x7fec900a84d0 2024-05-21T17:42:46.950 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.948+0000 7fec86ffd640 20 mgrc handle_mgr_map mgrmap(e 13) 2024-05-21T17:42:46.950 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.948+0000 7fec86ffd640 4 mgrc handle_mgr_map Got map version 13 2024-05-21T17:42:46.950 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.948+0000 7fec86ffd640 4 mgrc handle_mgr_map Active mgr is now 2024-05-21T17:42:46.950 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.948+0000 7fec86ffd640 4 mgrc reconnect Terminating session with v2:172.21.15.203:6824/4207378841 2024-05-21T17:42:46.950 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.948+0000 7fec86ffd640 1 -- 172.21.15.3:0/3179644757 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fec700af380 msgr2=0x7fec700b1840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-21T17:42:46.951 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.948+0000 7fec86ffd640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fec700af380 0x7fec700b1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:46.951 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:46.948+0000 7fec86ffd640 4 mgrc reconnect No active mgr available yet 2024-05-21T17:42:46.996 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.994+0000 7fc7cd243640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:46.996 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.994+0000 7fc7cd243640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:46.996 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.994+0000 7fc7cd243640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:46.996 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.994+0000 7fc7cd243640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:46.996 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.994+0000 7fc7cd243640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:46.997 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.995+0000 7fc7d0249640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:46.997 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.995+0000 7fc7d0249640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:46.997 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.995+0000 7fc7d0249640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:46.997 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.995+0000 7fc7d0249640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:46.997 INFO:tasks.ceph.mgr.z.smithi203.stderr:2024-05-21T17:42:46.995+0000 7fc7d0249640 -1 client.0 error registering admin socket command: (17) File exists 2024-05-21T17:42:47.038 INFO:tasks.ceph.mgr.x.smithi003.stderr:2024-05-21T17:42:47.037+0000 7f9dbba13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-21T17:42:47.040 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.039+0000 7fec94fe8640 1 -- 172.21.15.3:0/3179644757 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/failure_domain"} v 0) -- 0x7fec9006f910 con 0x7fec900a84d0 2024-05-21T17:42:47.046 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.045+0000 7fec86ffd640 1 -- 172.21.15.3:0/3179644757 <== mon.2 v2:172.21.15.3:3301/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/failure_domain"}]=0 v9) ==== 111+0+0 (secure 0 0 0) 0x7fec800c8070 con 0x7fec900a84d0 2024-05-21T17:42:47.158 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.157+0000 7fec94fe8640 1 -- 172.21.15.3:0/3179644757 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/failure_domain"} v 0) -- 0x7fec900008d0 con 0x7fec900a84d0 2024-05-21T17:42:47.158 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.157+0000 7fec86ffd640 1 -- 172.21.15.3:0/3179644757 <== mon.2 v2:172.21.15.3:3301/0 9 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/failure_domain"}]=0 v9) ==== 111+0+0 (secure 0 0 0) 0x7fec800a02e0 con 0x7fec900a84d0 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 10 mgrc shutdown 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 1 -- 172.21.15.3:0/3179644757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 msgr2=0x7fec900a4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a4060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec8002b930 tx=0x7fec800046c0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 1 -- 172.21.15.3:0/3179644757 shutdown_connections 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.203:6824/4207378841,v1:172.21.15.203:6825/4207378841] conn(0x7fec700af380 0x7fec700b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec9014a520 0x7fec9009f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec9014a140 0x7fec9009ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 1 --2- 172.21.15.3:0/3179644757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fec900a84d0 0x7fec900a4060 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 1 -- 172.21.15.3:0/3179644757 >> 172.21.15.3:0/3179644757 conn(0x7fec90075a50 msgr2=0x7fec900b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 1 -- 172.21.15.3:0/3179644757 shutdown_connections 2024-05-21T17:42:47.162 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-21T17:42:47.159+0000 7fec94fe8640 1 -- 172.21.15.3:0/3179644757 wait complete. 2024-05-21T17:42:47.170 INFO:tasks.workunit.client.0.smithi003.stderr:+ echo OK 2024-05-21T17:42:47.170 INFO:tasks.workunit.client.0.smithi003.stdout:OK 2024-05-21T17:42:47.171 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-21T17:42:47.171 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-21T17:42:47.237 INFO:tasks.workunit:Stopping ['mgr/test_localpool.sh'] on client.0... 2024-05-21T17:42:47.237 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-21T17:42:47.532 DEBUG:teuthology.parallel:result is None 2024-05-21T17:42:47.532 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-21T17:42:47.558 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-21T17:42:47.558 DEBUG:teuthology.orchestra.run.smithi003:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-21T17:42:47.627 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-21T17:42:47.627 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-21T17:42:47.636 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-21T17:42:47.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.768+0000 7fa0a6102640 1 Processor -- start 2024-05-21T17:42:47.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.768+0000 7fa0a6102640 1 -- start start 2024-05-21T17:42:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa0a6102640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a00a6d10 0x7fa0a014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa0a6102640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0a00a7180 0x7fa0a014ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa0a6102640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a00a7990 con 0x7fa0a0144cb0 2024-05-21T17:42:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa0a6102640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fa0a01440d0 con 0x7fa0a00a7180 2024-05-21T17:42:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa0a6102640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a00a6b00 con 0x7fa0a00a6d10 2024-05-21T17:42:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa09effd640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a00a6d10 0x7fa0a014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa09f7fe640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa0a0144cb0 0x7fa0a0073e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38480/0 (socket says 172.21.15.3:38480) 2024-05-21T17:42:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa09f7fe640 1 -- 172.21.15.3:0/4271020295 learned_addr learned my addr 172.21.15.3:0/4271020295 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:42:47.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa09ffff640 1 --2- 172.21.15.3:0/4271020295 >> v2:172.21.15.203:3300/0 conn(0x7fa0a00a7180 0x7fa0a014ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa09effd640 1 -- 172.21.15.3:0/4271020295 >> v2:172.21.15.203:3300/0 conn(0x7fa0a00a7180 msgr2=0x7fa0a014ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.770+0000 7fa09effd640 1 --2- 172.21.15.3:0/4271020295 >> v2:172.21.15.203:3300/0 conn(0x7fa0a00a7180 0x7fa0a014ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.769+0000 7fa09e7fc640 1 -- 172.21.15.3:0/4271020295 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 740722711 0 0) 0x7fa0a00a7990 con 0x7fa0a0144cb0 2024-05-21T17:42:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.770+0000 7fa09effd640 1 -- 172.21.15.3:0/4271020295 >> v1:172.21.15.3:6789/0 conn(0x7fa0a0144cb0 legacy=0x7fa0a0073e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.770+0000 7fa09effd640 1 -- 172.21.15.3:0/4271020295 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a0074550 con 0x7fa0a00a6d10 2024-05-21T17:42:47.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.770+0000 7fa09effd640 1 --2- 172.21.15.3:0/4271020295 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a00a6d10 0x7fa0a014c9d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa0940097a0 tx=0x7fa09402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b4ea2b02200d73 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:47.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.770+0000 7fa09e7fc640 1 -- 172.21.15.3:0/4271020295 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0940315a0 con 0x7fa0a00a6d10 2024-05-21T17:42:47.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.770+0000 7fa09e7fc640 1 -- 172.21.15.3:0/4271020295 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa094037070 con 0x7fa0a00a6d10 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.771+0000 7fa09e7fc640 1 -- 172.21.15.3:0/4271020295 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094031960 con 0x7fa0a00a6d10 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.771+0000 7fa0a6102640 1 -- 172.21.15.3:0/4271020295 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a00a6d10 msgr2=0x7fa0a014c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.771+0000 7fa0a6102640 1 --2- 172.21.15.3:0/4271020295 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a00a6d10 0x7fa0a014c9d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa0940097a0 tx=0x7fa09402b840 comp rx=0 tx=0).stop 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.771+0000 7fa0a6102640 1 -- 172.21.15.3:0/4271020295 shutdown_connections 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.771+0000 7fa0a6102640 1 --2- 172.21.15.3:0/4271020295 >> v2:172.21.15.203:3300/0 conn(0x7fa0a00a7180 0x7fa0a014ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.771+0000 7fa0a6102640 1 --2- 172.21.15.3:0/4271020295 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a00a6d10 0x7fa0a014c9d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.771+0000 7fa0a6102640 1 -- 172.21.15.3:0/4271020295 >> 172.21.15.3:0/4271020295 conn(0x7fa0a00a8c40 msgr2=0x7fa0a00a9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.771+0000 7fa0a6102640 1 -- 172.21.15.3:0/4271020295 shutdown_connections 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.771+0000 7fa0a6102640 1 -- 172.21.15.3:0/4271020295 wait complete. 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.772+0000 7fa0a6102640 1 Processor -- start 2024-05-21T17:42:47.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.772+0000 7fa0a6102640 1 -- start start 2024-05-21T17:42:47.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa0a6102640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a00a6d10 0x7fa0a0147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:47.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa0a6102640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a00a7180 0x7fa0a00a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:47.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa0a6102640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a0144cb0 0x7fa0a00a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:47.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa0a6102640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01445f0 con 0x7fa0a00a7180 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa0a6102640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0144330 con 0x7fa0a00a6d10 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa0a6102640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a00739c0 con 0x7fa0a0144cb0 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a00a7180 0x7fa0a00a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09ffff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a0144cb0 0x7fa0a00a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09ffff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a0144cb0 0x7fa0a00a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51614/0 (socket says 172.21.15.3:51614) 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09ffff640 1 -- 172.21.15.3:0/2180373882 learned_addr learned my addr 172.21.15.3:0/2180373882 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09effd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a00a7180 0x7fa0a00a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40666/0 (socket says 172.21.15.3:40666) 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09f7fe640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a00a6d10 0x7fa0a0147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09ffff640 1 -- 172.21.15.3:0/2180373882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a00a6d10 msgr2=0x7fa0a0147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09ffff640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a00a6d10 0x7fa0a0147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09ffff640 1 -- 172.21.15.3:0/2180373882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a00a7180 msgr2=0x7fa0a00a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09ffff640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a00a7180 0x7fa0a00a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09ffff640 1 -- 172.21.15.3:0/2180373882 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a006ea50 con 0x7fa0a0144cb0 2024-05-21T17:42:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.773+0000 7fa09ffff640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a0144cb0 0x7fa0a00a0b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa098007d20 tx=0x7fa09800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.774+0000 7fa087fff640 1 -- 172.21.15.3:0/2180373882 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098014420 con 0x7fa0a0144cb0 2024-05-21T17:42:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.774+0000 7fa087fff640 1 -- 172.21.15.3:0/2180373882 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa098019070 con 0x7fa0a0144cb0 2024-05-21T17:42:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.774+0000 7fa0a6102640 1 -- 172.21.15.3:0/2180373882 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a006ed10 con 0x7fa0a0144cb0 2024-05-21T17:42:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.774+0000 7fa087fff640 1 -- 172.21.15.3:0/2180373882 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098014820 con 0x7fa0a0144cb0 2024-05-21T17:42:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.774+0000 7fa0a6102640 1 -- 172.21.15.3:0/2180373882 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a009f1d0 con 0x7fa0a0144cb0 2024-05-21T17:42:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.774+0000 7fa0a6102640 1 -- 172.21.15.3:0/2180373882 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a0069290 con 0x7fa0a0144cb0 2024-05-21T17:42:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.776+0000 7fa087fff640 1 -- 172.21.15.3:0/2180373882 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 13) ==== 146702+0+0 (secure 0 0 0) 0x7fa0980149c0 con 0x7fa0a0144cb0 2024-05-21T17:42:47.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.776+0000 7fa087fff640 20 mgrc handle_mgr_map mgrmap(e 13) 2024-05-21T17:42:47.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.777+0000 7fa087fff640 4 mgrc handle_mgr_map Got map version 13 2024-05-21T17:42:47.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.777+0000 7fa087fff640 4 mgrc handle_mgr_map Active mgr is now 2024-05-21T17:42:47.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.777+0000 7fa087fff640 4 mgrc reconnect No active mgr available yet 2024-05-21T17:42:47.779 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.777+0000 7fa087fff640 1 -- 172.21.15.3:0/2180373882 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 5198+0+0 (secure 0 0 0) 0x7fa0980ddd00 con 0x7fa0a0144cb0 2024-05-21T17:42:47.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.780+0000 7fa087fff640 1 -- 172.21.15.3:0/2180373882 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa0980a7ba0 con 0x7fa0a0144cb0 2024-05-21T17:42:47.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.953+0000 7fa087fff640 1 -- 172.21.15.3:0/2180373882 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mgrmap(e 14) ==== 146829+0+0 (secure 0 0 0) 0x7fa0980a2d80 con 0x7fa0a0144cb0 2024-05-21T17:42:47.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.953+0000 7fa087fff640 20 mgrc handle_mgr_map mgrmap(e 14) 2024-05-21T17:42:47.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.954+0000 7fa087fff640 4 mgrc handle_mgr_map Got map version 14 2024-05-21T17:42:47.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.954+0000 7fa087fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:47.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.954+0000 7fa087fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:47.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.954+0000 7fa087fff640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa08c0b46d0 0x7fa08c0b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:47.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.954+0000 7fa09f7fe640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa08c0b46d0 0x7fa08c0b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:47.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.957+0000 7fa09f7fe640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa08c0b46d0 0x7fa08c0b6ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa090000f60 tx=0x7fa090007cf0 comp rx=0 tx=0).ready entity=mgr.4507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:47.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.975+0000 7fa0a6102640 1 -- 172.21.15.3:0/2180373882 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fa0a00ba5d0 con 0x7fa0a0144cb0 2024-05-21T17:42:47.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.979+0000 7fa087fff640 1 -- 172.21.15.3:0/2180373882 <== mon.2 v2:172.21.15.3:3301/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa0980a6fb0 con 0x7fa0a0144cb0 2024-05-21T17:42:47.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.979+0000 7fa087fff640 1 -- 172.21.15.3:0/2180373882 <== mon.2 v2:172.21.15.3:3301/0 9 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7fa09809e2c0 con 0x7fa0a0144cb0 2024-05-21T17:42:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.981+0000 7fa0a6102640 10 mgrc shutdown 2024-05-21T17:42:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 -- 172.21.15.3:0/2180373882 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa08c0b46d0 msgr2=0x7fa08c0b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa08c0b46d0 0x7fa08c0b6ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa090000f60 tx=0x7fa090007cf0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.983 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 -- 172.21.15.3:0/2180373882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a0144cb0 msgr2=0x7fa0a00a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a0144cb0 0x7fa0a00a0b80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa098007d20 tx=0x7fa09800e4b0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 -- 172.21.15.3:0/2180373882 shutdown_connections 2024-05-21T17:42:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa08c0b46d0 0x7fa08c0b6ac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa0a0144cb0 0x7fa0a00a0b80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0a00a7180 0x7fa0a00a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 --2- 172.21.15.3:0/2180373882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a00a6d10 0x7fa0a0147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 -- 172.21.15.3:0/2180373882 >> 172.21.15.3:0/2180373882 conn(0x7fa0a00a8c40 msgr2=0x7fa0a00b9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 -- 172.21.15.3:0/2180373882 shutdown_connections 2024-05-21T17:42:47.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:47.982+0000 7fa0a6102640 1 -- 172.21.15.3:0/2180373882 wait complete. 2024-05-21T17:42:47.994 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-21T17:42:47.994 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T17:42:48.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.099+0000 7fa1a665e640 1 Processor -- start 2024-05-21T17:42:48.100 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.099+0000 7fa1a665e640 1 -- start start 2024-05-21T17:42:48.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.099+0000 7fa1a665e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a01387f0 0x7fa1a00ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.099+0000 7fa1a665e640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a0138c80 con 0x7fa1a00abf70 2024-05-21T17:42:48.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.099+0000 7fa1a665e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a0138f20 con 0x7fa1a00ad6d0 2024-05-21T17:42:48.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.099+0000 7fa1a665e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a0139810 con 0x7fa1a01387f0 2024-05-21T17:42:48.101 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.100+0000 7fa19ffff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fa1a00abf70 0x7fa1a006c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38494/0 (socket says 172.21.15.3:38494) 2024-05-21T17:42:48.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.100+0000 7fa19ffff640 1 -- 172.21.15.3:0/3215929888 learned_addr learned my addr 172.21.15.3:0/3215929888 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:42:48.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.100+0000 7fa19f7fe640 1 --2- 172.21.15.3:0/3215929888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a01387f0 0x7fa1a00ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.100+0000 7fa19f7fe640 1 -- 172.21.15.3:0/3215929888 >> v1:172.21.15.203:6789/0 conn(0x7fa1a00ad6d0 legacy=0x7fa1a0150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.100+0000 7fa19effd640 1 -- 172.21.15.3:0/3215929888 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 810672368 0 0) 0x7fa1a0138c80 con 0x7fa1a00abf70 2024-05-21T17:42:48.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.100+0000 7fa19f7fe640 1 -- 172.21.15.3:0/3215929888 >> v1:172.21.15.3:6789/0 conn(0x7fa1a00abf70 legacy=0x7fa1a006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.102 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.100+0000 7fa19f7fe640 1 -- 172.21.15.3:0/3215929888 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a013c970 con 0x7fa1a01387f0 2024-05-21T17:42:48.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa19f7fe640 1 --2- 172.21.15.3:0/3215929888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a01387f0 0x7fa1a00ac350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1940097d0 tx=0x7fa19402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d78e1ce807e280a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:48.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa19effd640 1 -- 172.21.15.3:0/3215929888 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194035020 con 0x7fa1a01387f0 2024-05-21T17:42:48.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa19effd640 1 -- 172.21.15.3:0/3215929888 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa194031800 con 0x7fa1a01387f0 2024-05-21T17:42:48.103 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa19effd640 1 -- 172.21.15.3:0/3215929888 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194031bf0 con 0x7fa1a01387f0 2024-05-21T17:42:48.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa1a665e640 1 -- 172.21.15.3:0/3215929888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a01387f0 msgr2=0x7fa1a00ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa1a665e640 1 --2- 172.21.15.3:0/3215929888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a01387f0 0x7fa1a00ac350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1940097d0 tx=0x7fa19402b8a0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa1a665e640 1 -- 172.21.15.3:0/3215929888 shutdown_connections 2024-05-21T17:42:48.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa1a665e640 1 --2- 172.21.15.3:0/3215929888 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a01387f0 0x7fa1a00ac350 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa1a665e640 1 -- 172.21.15.3:0/3215929888 >> 172.21.15.3:0/3215929888 conn(0x7fa1a00aab70 msgr2=0x7fa1a00aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:48.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa1a665e640 1 -- 172.21.15.3:0/3215929888 shutdown_connections 2024-05-21T17:42:48.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.101+0000 7fa1a665e640 1 -- 172.21.15.3:0/3215929888 wait complete. 2024-05-21T17:42:48.104 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.102+0000 7fa1a665e640 1 Processor -- start 2024-05-21T17:42:48.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa1a665e640 1 -- start start 2024-05-21T17:42:48.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa1a665e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00abf70 0x7fa1a00b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa1a665e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a00ad6d0 0x7fa1a01381b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa1a665e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a00a7ab0 0x7fa1a014f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa1a665e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a00adb00 con 0x7fa1a00abf70 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa1a665e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a00a52e0 con 0x7fa1a00ad6d0 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa1a665e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a00adc80 con 0x7fa1a00a7ab0 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa19ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00abf70 0x7fa1a00b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa19ffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00abf70 0x7fa1a00b8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40668/0 (socket says 172.21.15.3:40668) 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa19ffff640 1 -- 172.21.15.3:0/1795649757 learned_addr learned my addr 172.21.15.3:0/1795649757 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.103+0000 7fa1a4bfb640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a00a7ab0 0x7fa1a014f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19f7fe640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a00ad6d0 0x7fa1a01381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19ffff640 1 -- 172.21.15.3:0/1795649757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a00a7ab0 msgr2=0x7fa1a014f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19ffff640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a00a7ab0 0x7fa1a014f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.107 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19ffff640 1 -- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a00ad6d0 msgr2=0x7fa1a01381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19ffff640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a00ad6d0 0x7fa1a01381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19ffff640 1 -- 172.21.15.3:0/1795649757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a0150da0 con 0x7fa1a00abf70 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19f7fe640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a00ad6d0 0x7fa1a01381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa1a4bfb640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a00a7ab0 0x7fa1a014f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19ffff640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00abf70 0x7fa1a00b8960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa19000c3d0 tx=0x7fa19000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19cff9640 1 -- 172.21.15.3:0/1795649757 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900074e0 con 0x7fa1a00abf70 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19cff9640 1 -- 172.21.15.3:0/1795649757 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa1900047e0 con 0x7fa1a00abf70 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa19cff9640 1 -- 172.21.15.3:0/1795649757 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190004bf0 con 0x7fa1a00abf70 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa1a665e640 1 -- 172.21.15.3:0/1795649757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a013adc0 con 0x7fa1a00abf70 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.104+0000 7fa1a665e640 1 -- 172.21.15.3:0/1795649757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a014f220 con 0x7fa1a00abf70 2024-05-21T17:42:48.108 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.105+0000 7fa1a665e640 1 -- 172.21.15.3:0/1795649757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a014f4e0 con 0x7fa1a00abf70 2024-05-21T17:42:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.106+0000 7fa19cff9640 1 -- 172.21.15.3:0/1795649757 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) ==== 146829+0+0 (secure 0 0 0) 0x7fa190011070 con 0x7fa1a00abf70 2024-05-21T17:42:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.106+0000 7fa19cff9640 20 mgrc handle_mgr_map mgrmap(e 14) 2024-05-21T17:42:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.107+0000 7fa19cff9640 4 mgrc handle_mgr_map Got map version 14 2024-05-21T17:42:48.109 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.107+0000 7fa19cff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.107+0000 7fa19cff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.107+0000 7fa19cff9640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa18c0af0b0 0x7fa18c0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.107+0000 7fa19cff9640 1 -- 172.21.15.3:0/1795649757 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5198+0+0 (secure 0 0 0) 0x7fa1900dc580 con 0x7fa1a00abf70 2024-05-21T17:42:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.107+0000 7fa19f7fe640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa18c0af0b0 0x7fa18c0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.108+0000 7fa19f7fe640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa18c0af0b0 0x7fa18c0b1570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa194036000 tx=0x7fa194008030 comp rx=0 tx=0).ready entity=mgr.4507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:48.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.111+0000 7fa19cff9640 1 -- 172.21.15.3:0/1795649757 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa1a014f4e0 con 0x7fa1a00abf70 2024-05-21T17:42:48.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.345+0000 7fa1a665e640 1 -- 172.21.15.3:0/1795649757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1a0069a90 con 0x7fa1a00abf70 2024-05-21T17:42:48.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.345+0000 7fa19cff9640 1 -- 172.21.15.3:0/1795649757 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+4345 (secure 0 0 0) 0x7fa1900a6240 con 0x7fa1a00abf70 2024-05-21T17:42:48.347 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:42:48.347 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/3741447020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3741447020},{"type":"v1","addr":"172.21.15.203:6827","nonce":3741447020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6810/205822275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":205822275},{"type":"v1","addr":"172.21.15.3:6810","nonce":205822275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-21T17:41:32.553094+0000","modified":"2024-05-21T17:41:35.579599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6811/1903526925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6809","nonce":1903526925},{"type":"v1","addr":"172.21.15.3:6811","nonce":1903526925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-05-21T17:42:48.347 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-21T17:42:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.347+0000 7fa1a665e640 10 mgrc shutdown 2024-05-21T17:42:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 -- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa18c0af0b0 msgr2=0x7fa18c0b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa18c0af0b0 0x7fa18c0b1570 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa194036000 tx=0x7fa194008030 comp rx=0 tx=0).stop 2024-05-21T17:42:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 -- 172.21.15.3:0/1795649757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00abf70 msgr2=0x7fa1a00b8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00abf70 0x7fa1a00b8960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa19000c3d0 tx=0x7fa19000c890 comp rx=0 tx=0).stop 2024-05-21T17:42:48.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 -- 172.21.15.3:0/1795649757 shutdown_connections 2024-05-21T17:42:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fa18c0af0b0 0x7fa18c0b1570 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fa1a00a7ab0 0x7fa1a014f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1a00ad6d0 0x7fa1a01381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 --2- 172.21.15.3:0/1795649757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa1a00abf70 0x7fa1a00b8960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 -- 172.21.15.3:0/1795649757 >> 172.21.15.3:0/1795649757 conn(0x7fa1a00aab70 msgr2=0x7fa1a00a8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 -- 172.21.15.3:0/1795649757 shutdown_connections 2024-05-21T17:42:48.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.348+0000 7fa1a665e640 1 -- 172.21.15.3:0/1795649757 wait complete. 2024-05-21T17:42:48.360 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-21T17:42:48.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.462+0000 7ff970fed640 1 Processor -- start 2024-05-21T17:42:48.463 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.462+0000 7ff970fed640 1 -- start start 2024-05-21T17:42:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff970fed640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a6c60 0x7ff96c0a7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff970fed640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff96c0a8020 0x7ff96c1468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff970fed640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff96c0a8440 0x7ff96c146df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.464 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff970fed640 1 -- --> v2:172.21.15.3:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c0a7840 con 0x7ff96c0a8020 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff970fed640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c1465f0 con 0x7ff96c0a8440 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff970fed640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c073870 con 0x7ff96c0a6c60 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a6c60 0x7ff96c0a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff963fff640 1 --2- >> v2:172.21.15.3:3300/0 conn(0x7ff96c0a8020 0x7ff96c1468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a6c60 0x7ff96c0a7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51630/0 (socket says 172.21.15.3:51630) 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff96a59c640 1 -- 172.21.15.3:0/2507202268 learned_addr learned my addr 172.21.15.3:0/2507202268 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff96a59c640 1 -- 172.21.15.3:0/2507202268 >> v2:172.21.15.203:3300/0 conn(0x7ff96c0a8440 msgr2=0x7ff96c146df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff96a59c640 1 --2- 172.21.15.3:0/2507202268 >> v2:172.21.15.203:3300/0 conn(0x7ff96c0a8440 0x7ff96c146df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff96a59c640 1 -- 172.21.15.3:0/2507202268 >> v2:172.21.15.3:3300/0 conn(0x7ff96c0a8020 msgr2=0x7ff96c1468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff96a59c640 1 --2- 172.21.15.3:0/2507202268 >> v2:172.21.15.3:3300/0 conn(0x7ff96c0a8020 0x7ff96c1468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff96a59c640 1 -- 172.21.15.3:0/2507202268 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c146260 con 0x7ff96c0a6c60 2024-05-21T17:42:48.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.463+0000 7ff963fff640 1 --2- 172.21.15.3:0/2507202268 >> v2:172.21.15.3:3300/0 conn(0x7ff96c0a8020 0x7ff96c1468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff96a59c640 1 --2- 172.21.15.3:0/2507202268 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a6c60 0x7ff96c0a7040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff96c076190 tx=0x7ff95c02b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da830499353d5180 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff969d9b640 1 -- 172.21.15.3:0/2507202268 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c038070 con 0x7ff96c0a6c60 2024-05-21T17:42:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff969d9b640 1 -- 172.21.15.3:0/2507202268 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff95c031740 con 0x7ff96c0a6c60 2024-05-21T17:42:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff969d9b640 1 -- 172.21.15.3:0/2507202268 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c02f050 con 0x7ff96c0a6c60 2024-05-21T17:42:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff970fed640 1 -- 172.21.15.3:0/2507202268 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a6c60 msgr2=0x7ff96c0a7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff970fed640 1 --2- 172.21.15.3:0/2507202268 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a6c60 0x7ff96c0a7040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff96c076190 tx=0x7ff95c02b7e0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff970fed640 1 -- 172.21.15.3:0/2507202268 shutdown_connections 2024-05-21T17:42:48.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff970fed640 1 --2- 172.21.15.3:0/2507202268 >> v2:172.21.15.203:3300/0 conn(0x7ff96c0a8440 0x7ff96c146df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff970fed640 1 --2- 172.21.15.3:0/2507202268 >> v2:172.21.15.3:3300/0 conn(0x7ff96c0a8020 0x7ff96c1468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff970fed640 1 --2- 172.21.15.3:0/2507202268 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a6c60 0x7ff96c0a7040 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.464+0000 7ff970fed640 1 -- 172.21.15.3:0/2507202268 >> 172.21.15.3:0/2507202268 conn(0x7ff96c0a8ad0 msgr2=0x7ff96c0a8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.465+0000 7ff970fed640 1 -- 172.21.15.3:0/2507202268 shutdown_connections 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.465+0000 7ff970fed640 1 -- 172.21.15.3:0/2507202268 wait complete. 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff970fed640 1 Processor -- start 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff970fed640 1 -- start start 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff970fed640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0a6c60 0x7ff96c1cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff970fed640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a8020 0x7ff96c1d1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff970fed640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c0a8440 0x7ff96c1d2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff970fed640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c0a6a50 con 0x7ff96c0a6c60 2024-05-21T17:42:48.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff970fed640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c0a68d0 con 0x7ff96c0a8440 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff970fed640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c149820 con 0x7ff96c0a8020 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff963fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a8020 0x7ff96c1d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff96a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0a6c60 0x7ff96c1cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff963fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a8020 0x7ff96c1d1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51636/0 (socket says 172.21.15.3:51636) 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.466+0000 7ff963fff640 1 -- 172.21.15.3:0/2547298320 learned_addr learned my addr 172.21.15.3:0/2547298320 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff963fff640 1 -- 172.21.15.3:0/2547298320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c0a8440 msgr2=0x7ff96c1d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff96ad9d640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c0a8440 0x7ff96c1d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff963fff640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c0a8440 0x7ff96c1d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff963fff640 1 -- 172.21.15.3:0/2547298320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0a6c60 msgr2=0x7ff96c1cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff963fff640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0a6c60 0x7ff96c1cfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff963fff640 1 -- 172.21.15.3:0/2547298320 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c0a4fb0 con 0x7ff96c0a8020 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff96a59c640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0a6c60 0x7ff96c1cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:48.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff963fff640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a8020 0x7ff96c1d1810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff95400ebe0 tx=0x7ff95400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff962ffd640 1 -- 172.21.15.3:0/2547298320 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954017070 con 0x7ff96c0a8020 2024-05-21T17:42:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff962ffd640 1 -- 172.21.15.3:0/2547298320 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff95400cc30 con 0x7ff96c0a8020 2024-05-21T17:42:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff970fed640 1 -- 172.21.15.3:0/2547298320 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c1d2810 con 0x7ff96c0a8020 2024-05-21T17:42:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff962ffd640 1 -- 172.21.15.3:0/2547298320 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954013470 con 0x7ff96c0a8020 2024-05-21T17:42:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.467+0000 7ff970fed640 1 -- 172.21.15.3:0/2547298320 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c0a5b20 con 0x7ff96c0a8020 2024-05-21T17:42:48.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.468+0000 7ff970fed640 1 -- 172.21.15.3:0/2547298320 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c069a90 con 0x7ff96c0a8020 2024-05-21T17:42:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.469+0000 7ff962ffd640 1 -- 172.21.15.3:0/2547298320 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 14) ==== 146829+0+0 (secure 0 0 0) 0x7ff95401a070 con 0x7ff96c0a8020 2024-05-21T17:42:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.469+0000 7ff962ffd640 20 mgrc handle_mgr_map mgrmap(e 14) 2024-05-21T17:42:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.470+0000 7ff962ffd640 4 mgrc handle_mgr_map Got map version 14 2024-05-21T17:42:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.470+0000 7ff962ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.470+0000 7ff962ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.470+0000 7ff962ffd640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7ff94c0af0b0 0x7ff94c0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.470+0000 7ff962ffd640 1 -- 172.21.15.3:0/2547298320 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 5198+0+0 (secure 0 0 0) 0x7ff9540dd020 con 0x7ff96c0a8020 2024-05-21T17:42:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.471+0000 7ff96a59c640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7ff94c0af0b0 0x7ff94c0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.471+0000 7ff96a59c640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7ff94c0af0b0 0x7ff94c0b1570 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff95c035000 tx=0x7ff95c008030 comp rx=0 tx=0).ready entity=mgr.4507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:48.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.473+0000 7ff962ffd640 1 -- 172.21.15.3:0/2547298320 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff9540a6db0 con 0x7ff96c0a8020 2024-05-21T17:42:48.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.700+0000 7ff970fed640 1 -- 172.21.15.3:0/2547298320 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff96c073ad0 con 0x7ff96c0a8020 2024-05-21T17:42:48.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.701+0000 7ff962ffd640 1 -- 172.21.15.3:0/2547298320 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+4345 (secure 0 0 0) 0x7ff9540a66b0 con 0x7ff96c0a8020 2024-05-21T17:42:48.702 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:42:48.702 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6827/3741447020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":3741447020},{"type":"v1","addr":"172.21.15.203:6827","nonce":3741447020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6810/205822275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":205822275},{"type":"v1","addr":"172.21.15.3:6810","nonce":205822275}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-21T17:41:32.553094+0000","modified":"2024-05-21T17:41:35.579599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.3:6811/1903526925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6809","nonce":1903526925},{"type":"v1","addr":"172.21.15.3:6811","nonce":1903526925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4245,"qdb_cluster":[4245]},"id":1}]} 2024-05-21T17:42:48.703 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-21T17:42:48.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 10 mgrc shutdown 2024-05-21T17:42:48.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 -- 172.21.15.3:0/2547298320 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7ff94c0af0b0 msgr2=0x7ff94c0b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7ff94c0af0b0 0x7ff94c0b1570 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff95c035000 tx=0x7ff95c008030 comp rx=0 tx=0).stop 2024-05-21T17:42:48.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 -- 172.21.15.3:0/2547298320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a8020 msgr2=0x7ff96c1d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a8020 0x7ff96c1d1810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff95400ebe0 tx=0x7ff95400c520 comp rx=0 tx=0).stop 2024-05-21T17:42:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 -- 172.21.15.3:0/2547298320 shutdown_connections 2024-05-21T17:42:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7ff94c0af0b0 0x7ff94c0b1570 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff96c0a8440 0x7ff96c1d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7ff96c0a8020 0x7ff96c1d1810 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 --2- 172.21.15.3:0/2547298320 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff96c0a6c60 0x7ff96c1cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 -- 172.21.15.3:0/2547298320 >> 172.21.15.3:0/2547298320 conn(0x7ff96c0a8ad0 msgr2=0x7ff96c0b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 -- 172.21.15.3:0/2547298320 shutdown_connections 2024-05-21T17:42:48.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.703+0000 7ff970fed640 1 -- 172.21.15.3:0/2547298320 wait complete. 2024-05-21T17:42:48.716 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-21T17:42:48.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.819+0000 7fd90afe5640 1 Processor -- start 2024-05-21T17:42:48.820 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.819+0000 7fd90afe5640 1 -- start start 2024-05-21T17:42:48.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.819+0000 7fd90afe5640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd90414d800 0x7fd90414fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.819+0000 7fd90afe5640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9040a3150 con 0x7fd904074750 2024-05-21T17:42:48.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.819+0000 7fd90afe5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9041453d0 con 0x7fd9040a1720 2024-05-21T17:42:48.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.819+0000 7fd90afe5640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fd9040a2070 con 0x7fd90414d800 2024-05-21T17:42:48.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.819+0000 7fd909582640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd90414d800 0x7fd90414fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.819+0000 7fd903fff640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fd904074750 0x7fd904146ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38498/0 (socket says 172.21.15.3:38498) 2024-05-21T17:42:48.821 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.819+0000 7fd903fff640 1 -- 172.21.15.3:0/945743419 learned_addr learned my addr 172.21.15.3:0/945743419 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:42:48.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.820+0000 7fd9037fe640 1 -- 172.21.15.3:0/945743419 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3390512704 0 0) 0x7fd9040a3150 con 0x7fd904074750 2024-05-21T17:42:48.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.820+0000 7fd909582640 1 -- 172.21.15.3:0/945743419 >> v1:172.21.15.203:6789/0 conn(0x7fd9040a1720 legacy=0x7fd9040a2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:48.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.820+0000 7fd909582640 1 -- 172.21.15.3:0/945743419 >> v1:172.21.15.3:6789/0 conn(0x7fd904074750 legacy=0x7fd904146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.820+0000 7fd909582640 1 -- 172.21.15.3:0/945743419 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90406ead0 con 0x7fd90414d800 2024-05-21T17:42:48.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd909582640 1 --2- 172.21.15.3:0/945743419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd90414d800 0x7fd90414fbf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009dc0 tx=0x7fd8fc02bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec44ca6bff7b8bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:48.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd9037fe640 1 -- 172.21.15.3:0/945743419 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc039070 con 0x7fd90414d800 2024-05-21T17:42:48.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd9037fe640 1 -- 172.21.15.3:0/945743419 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd8fc02cb90 con 0x7fd90414d800 2024-05-21T17:42:48.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd9037fe640 1 -- 172.21.15.3:0/945743419 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc034440 con 0x7fd90414d800 2024-05-21T17:42:48.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd90afe5640 1 -- 172.21.15.3:0/945743419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd90414d800 msgr2=0x7fd90414fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd90afe5640 1 --2- 172.21.15.3:0/945743419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd90414d800 0x7fd90414fbf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009dc0 tx=0x7fd8fc02bde0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd90afe5640 1 -- 172.21.15.3:0/945743419 shutdown_connections 2024-05-21T17:42:48.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd90afe5640 1 --2- 172.21.15.3:0/945743419 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd90414d800 0x7fd90414fbf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd90afe5640 1 -- 172.21.15.3:0/945743419 >> 172.21.15.3:0/945743419 conn(0x7fd90406ccf0 msgr2=0x7fd9040a6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:48.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd90afe5640 1 -- 172.21.15.3:0/945743419 shutdown_connections 2024-05-21T17:42:48.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.821+0000 7fd90afe5640 1 -- 172.21.15.3:0/945743419 wait complete. 2024-05-21T17:42:48.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.822+0000 7fd90afe5640 1 Processor -- start 2024-05-21T17:42:48.824 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.822+0000 7fd90afe5640 1 -- start start 2024-05-21T17:42:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd90afe5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd904074750 0x7fd904071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd90afe5640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9040a1720 0x7fd90413f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd90afe5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90414d800 0x7fd90413fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd90afe5640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd90406e050 con 0x7fd90414d800 2024-05-21T17:42:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd90afe5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd9040a1b00 con 0x7fd904074750 2024-05-21T17:42:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd90afe5640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fd904073080 con 0x7fd9040a1720 2024-05-21T17:42:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd903fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9040a1720 0x7fd90413f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd909582640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90414d800 0x7fd90413fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd909582640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90414d800 0x7fd90413fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40694/0 (socket says 172.21.15.3:40694) 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd903fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9040a1720 0x7fd90413f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51646/0 (socket says 172.21.15.3:51646) 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd909582640 1 -- 172.21.15.3:0/2821143379 learned_addr learned my addr 172.21.15.3:0/2821143379 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd903fff640 1 -- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd904074750 msgr2=0x7fd904071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd908d81640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd904074750 0x7fd904071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd903fff640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd904074750 0x7fd904071bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd903fff640 1 -- 172.21.15.3:0/2821143379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90414d800 msgr2=0x7fd90413fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.823+0000 7fd903fff640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90414d800 0x7fd90413fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.824+0000 7fd903fff640 1 -- 172.21.15.3:0/2821143379 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd904069370 con 0x7fd9040a1720 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.824+0000 7fd909582640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90414d800 0x7fd90413fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.824+0000 7fd908d81640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd904074750 0x7fd904071bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.824+0000 7fd903fff640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9040a1720 0x7fd90413f6e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c430 tx=0x7fd8f000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.824+0000 7fd9017fa640 1 -- 172.21.15.3:0/2821143379 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00074e0 con 0x7fd9040a1720 2024-05-21T17:42:48.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.824+0000 7fd9017fa640 1 -- 172.21.15.3:0/2821143379 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd8f00047e0 con 0x7fd9040a1720 2024-05-21T17:42:48.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.824+0000 7fd90afe5640 1 -- 172.21.15.3:0/2821143379 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9040729e0 con 0x7fd9040a1720 2024-05-21T17:42:48.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.824+0000 7fd9017fa640 1 -- 172.21.15.3:0/2821143379 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0004bd0 con 0x7fd9040a1720 2024-05-21T17:42:48.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.824+0000 7fd90afe5640 1 -- 172.21.15.3:0/2821143379 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90413c200 con 0x7fd9040a1720 2024-05-21T17:42:48.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.825+0000 7fd90afe5640 1 -- 172.21.15.3:0/2821143379 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd904069a90 con 0x7fd9040a1720 2024-05-21T17:42:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.826+0000 7fd9017fa640 1 -- 172.21.15.3:0/2821143379 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 14) ==== 146829+0+0 (secure 0 0 0) 0x7fd8f0011070 con 0x7fd9040a1720 2024-05-21T17:42:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.826+0000 7fd9017fa640 20 mgrc handle_mgr_map mgrmap(e 14) 2024-05-21T17:42:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.827+0000 7fd9017fa640 4 mgrc handle_mgr_map Got map version 14 2024-05-21T17:42:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.827+0000 7fd9017fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.827+0000 7fd9017fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.827+0000 7fd9017fa640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fd8e80af0b0 0x7fd8e80b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.827+0000 7fd9017fa640 1 -- 172.21.15.3:0/2821143379 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 5198+0+0 (secure 0 0 0) 0x7fd8f00dc530 con 0x7fd9040a1720 2024-05-21T17:42:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.827+0000 7fd908d81640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fd8e80af0b0 0x7fd8e80b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:48.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.828+0000 7fd908d81640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fd8e80af0b0 0x7fd8e80b1570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8003eb0 tx=0x7fd8f8009040 comp rx=0 tx=0).ready entity=mgr.4507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:48.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:48.831+0000 7fd9017fa640 1 -- 172.21.15.3:0/2821143379 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd8f00a62c0 con 0x7fd9040a1720 2024-05-21T17:42:49.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.024+0000 7fd90afe5640 1 -- 172.21.15.3:0/2821143379 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9040a3500 con 0x7fd9040a1720 2024-05-21T17:42:49.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.025+0000 7fd9017fa640 1 -- 172.21.15.3:0/2821143379 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+14652 (secure 0 0 0) 0x7fd8f00a5bc0 con 0x7fd9040a1720 2024-05-21T17:42:49.027 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-21T17:42:49.027 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":27,"fsid":"e6b0b7c3-d6d8-4ed5-9e0c-6af41a72b442","created":"2024-05-21T17:41:17.034337+0000","modified":"2024-05-21T17:42:46.934462+0000","last_up_change":"2024-05-21T17:41:25.136644+0000","last_in_change":"2024-05-21T17:41:18.179591+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-21T17:41:25.537561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-21T17:41:26.016407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-05-21T17:41:30.560078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-05-21T17:41:31.496059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.190000057220459,"score_stable":1.190000057220459,"optimal_score":1,"raw_score_acting":1.190000057220459,"raw_score_stable":1.190000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7ac7071-219d-4cf4-9f8a-977f495fd0d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6801","nonce":3296405141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6803","nonce":3296405141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6807","nonce":3296405141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3296405141},{"type":"v1","addr":"172.21.15.3:6805","nonce":3296405141}]},"public_addr":"172.21.15.3:6801/3296405141","cluster_addr":"172.21.15.3:6803/3296405141","heartbeat_back_addr":"172.21.15.3:6807/3296405141","heartbeat_front_addr":"172.21.15.3:6805/3296405141","state":["exists","up"]},{"osd":1,"uuid":"89df5105-5900-4ffa-b8c1-6f480865a085","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6801","nonce":1777872358}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6803","nonce":1777872358}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6810","nonce":1777872358}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":1777872358},{"type":"v1","addr":"172.21.15.203:6805","nonce":1777872358}]},"public_addr":"172.21.15.203:6801/1777872358","cluster_addr":"172.21.15.203:6803/1777872358","heartbeat_back_addr":"172.21.15.203:6810/1777872358","heartbeat_front_addr":"172.21.15.203:6805/1777872358","state":["exists","up"]},{"osd":2,"uuid":"15f83334-8efc-40c2-af0c-eeed45923086","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6817","nonce":1069390381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6819","nonce":1069390381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6823","nonce":1069390381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1069390381},{"type":"v1","addr":"172.21.15.203:6821","nonce":1069390381}]},"public_addr":"172.21.15.203:6817/1069390381","cluster_addr":"172.21.15.203:6819/1069390381","heartbeat_back_addr":"172.21.15.203:6823/1069390381","heartbeat_front_addr":"172.21.15.203:6821/1069390381","state":["exists","up"]},{"osd":3,"uuid":"6a27ac8e-f957-4455-b477-3090081766c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6808","nonce":1072938612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6809","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6811","nonce":1072938612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6815","nonce":1072938612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":1072938612},{"type":"v1","addr":"172.21.15.203:6813","nonce":1072938612}]},"public_addr":"172.21.15.203:6808/1072938612","cluster_addr":"172.21.15.203:6811/1072938612","heartbeat_back_addr":"172.21.15.203:6815/1072938612","heartbeat_front_addr":"172.21.15.203:6813/1072938612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.224040+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.344461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.319792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-21T17:41:23.332782+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.6","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.a","mappings":[{"from":3,"to":0}]},{"pgid":"4.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.203:0/2440832496":"2024-05-22T17:42:46.934402+0000","172.21.15.203:0/515614797":"2024-05-22T17:42:39.045884+0000","172.21.15.203:6824/4157202067":"2024-05-22T17:42:39.045884+0000","172.21.15.203:6825/4157202067":"2024-05-22T17:42:39.045884+0000","172.21.15.203:0/2038920480":"2024-05-22T17:42:46.934402+0000","172.21.15.203:0/2295098024":"2024-05-22T17:42:39.045884+0000","172.21.15.203:0/1911408330":"2024-05-22T17:42:39.045884+0000","172.21.15.203:6824/4207378841":"2024-05-22T17:42:46.934402+0000","172.21.15.203:0/2972207335":"2024-05-22T17:42:39.045884+0000","172.21.15.203:0/1074421508":"2024-05-22T17:42:46.934402+0000","172.21.15.203:0/1434156270":"2024-05-22T17:42:46.934402+0000","172.21.15.203:0/1331791965":"2024-05-22T17:42:46.934402+0000","172.21.15.203:0/4171500432":"2024-05-22T17:42:46.934402+0000","172.21.15.203:6825/4207378841":"2024-05-22T17:42:46.934402+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-21T17:42:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.027+0000 7fd90afe5640 10 mgrc shutdown 2024-05-21T17:42:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.027+0000 7fd90afe5640 1 -- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fd8e80af0b0 msgr2=0x7fd8e80b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.027+0000 7fd90afe5640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fd8e80af0b0 0x7fd8e80b1570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8003eb0 tx=0x7fd8f8009040 comp rx=0 tx=0).stop 2024-05-21T17:42:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.027+0000 7fd90afe5640 1 -- 172.21.15.3:0/2821143379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9040a1720 msgr2=0x7fd90413f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.027+0000 7fd90afe5640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9040a1720 0x7fd90413f6e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c430 tx=0x7fd8f000c8f0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.028+0000 7fd90afe5640 1 -- 172.21.15.3:0/2821143379 shutdown_connections 2024-05-21T17:42:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.028+0000 7fd90afe5640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fd8e80af0b0 0x7fd8e80b1570 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.028+0000 7fd90afe5640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90414d800 0x7fd90413fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.028+0000 7fd90afe5640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fd9040a1720 0x7fd90413f6e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.028+0000 7fd90afe5640 1 --2- 172.21.15.3:0/2821143379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd904074750 0x7fd904071bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.028+0000 7fd90afe5640 1 -- 172.21.15.3:0/2821143379 >> 172.21.15.3:0/2821143379 conn(0x7fd90406ccf0 msgr2=0x7fd90413e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.028+0000 7fd90afe5640 1 -- 172.21.15.3:0/2821143379 shutdown_connections 2024-05-21T17:42:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.028+0000 7fd90afe5640 1 -- 172.21.15.3:0/2821143379 wait complete. 2024-05-21T17:42:49.040 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-21T17:42:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.143+0000 7fc449f69640 1 Processor -- start 2024-05-21T17:42:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.143+0000 7fc449f69640 1 -- start start 2024-05-21T17:42:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.144+0000 7fc449f69640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc4440a7180 0x7fc444150100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.144+0000 7fc449f69640 1 -- --> v1:172.21.15.3:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4440a7990 con 0x7fc4440a6d10 2024-05-21T17:42:49.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.144+0000 7fc449f69640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc444074550 con 0x7fc444144cb0 2024-05-21T17:42:49.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.144+0000 7fc449f69640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7fc4441440d0 con 0x7fc4440a7180 2024-05-21T17:42:49.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.144+0000 7fc442ffd640 1 --1- >> v1:172.21.15.3:6789/0 conn(0x7fc4440a6d10 0x7fc44414c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.3:6789/0 says I am v1:172.21.15.3:38510/0 (socket says 172.21.15.3:38510) 2024-05-21T17:42:49.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.144+0000 7fc442ffd640 1 -- 172.21.15.3:0/3090850085 learned_addr learned my addr 172.21.15.3:0/3090850085 (peer_addr_for_me v1:172.21.15.3:0/0) 2024-05-21T17:42:49.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.144+0000 7fc443fff640 1 --2- 172.21.15.3:0/3090850085 >> v2:172.21.15.203:3300/0 conn(0x7fc4440a7180 0x7fc444150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.144+0000 7fc4427fc640 1 -- 172.21.15.3:0/3090850085 <== mon.0 v1:172.21.15.3:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3278833573 0 0) 0x7fc4440a7990 con 0x7fc4440a6d10 2024-05-21T17:42:49.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.144+0000 7fc4427fc640 1 -- 172.21.15.3:0/3090850085 --> v1:172.21.15.3:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc42c003520 con 0x7fc4440a6d10 2024-05-21T17:42:49.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.145+0000 7fc443fff640 1 -- 172.21.15.3:0/3090850085 >> v1:172.21.15.203:6789/0 conn(0x7fc444144cb0 legacy=0x7fc444073e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.145+0000 7fc443fff640 1 -- 172.21.15.3:0/3090850085 >> v1:172.21.15.3:6789/0 conn(0x7fc4440a6d10 legacy=0x7fc44414c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.149 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.145+0000 7fc4427fc640 1 -- 172.21.15.3:0/3090850085 <== mon.0 v1:172.21.15.3:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3648164782 0 0) 0x7fc42c003520 con 0x7fc4440a6d10 2024-05-21T17:42:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.145+0000 7fc443fff640 1 -- 172.21.15.3:0/3090850085 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4440a6b00 con 0x7fc4440a7180 2024-05-21T17:42:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.145+0000 7fc443fff640 1 --2- 172.21.15.3:0/3090850085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4440a7180 0x7fc444150100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009d90 tx=0x7fc43c02bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a70c6e0e25ebb296 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.146+0000 7fc4427fc640 1 -- 172.21.15.3:0/3090850085 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43c039070 con 0x7fc4440a7180 2024-05-21T17:42:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.146+0000 7fc4427fc640 1 -- 172.21.15.3:0/3090850085 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc43c02c8b0 con 0x7fc4440a7180 2024-05-21T17:42:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.146+0000 7fc4427fc640 1 -- 172.21.15.3:0/3090850085 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4380016f0 con 0x7fc4440a7180 2024-05-21T17:42:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.146+0000 7fc449f69640 1 -- 172.21.15.3:0/3090850085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4440a7180 msgr2=0x7fc444150100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.146+0000 7fc449f69640 1 --2- 172.21.15.3:0/3090850085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4440a7180 0x7fc444150100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009d90 tx=0x7fc43c02bd80 comp rx=0 tx=0).stop 2024-05-21T17:42:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.146+0000 7fc449f69640 1 -- 172.21.15.3:0/3090850085 shutdown_connections 2024-05-21T17:42:49.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.146+0000 7fc449f69640 1 --2- 172.21.15.3:0/3090850085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4440a7180 0x7fc444150100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.146+0000 7fc449f69640 1 -- 172.21.15.3:0/3090850085 >> 172.21.15.3:0/3090850085 conn(0x7fc4440a8c40 msgr2=0x7fc4440a9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.146+0000 7fc449f69640 1 -- 172.21.15.3:0/3090850085 shutdown_connections 2024-05-21T17:42:49.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.146+0000 7fc449f69640 1 -- 172.21.15.3:0/3090850085 wait complete. 2024-05-21T17:42:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.147+0000 7fc449f69640 1 Processor -- start 2024-05-21T17:42:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.148+0000 7fc449f69640 1 -- start start 2024-05-21T17:42:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.148+0000 7fc449f69640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4440a6d10 0x7fc4440a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.148+0000 7fc449f69640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4440a7180 0x7fc444140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.148+0000 7fc449f69640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc444144cb0 0x7fc4441cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.148+0000 7fc449f69640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc4440739c0 con 0x7fc4440a6d10 2024-05-21T17:42:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.148+0000 7fc449f69640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc4441445f0 con 0x7fc4440a7180 2024-05-21T17:42:49.154 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.148+0000 7fc449f69640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7fc4440749c0 con 0x7fc444144cb0 2024-05-21T17:42:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.148+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4440a6d10 0x7fc4440a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc4437fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4440a6d10 0x7fc4440a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40706/0 (socket says 172.21.15.3:40706) 2024-05-21T17:42:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc443fff640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc444144cb0 0x7fc4441cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc4437fe640 1 -- 172.21.15.3:0/2766234388 learned_addr learned my addr 172.21.15.3:0/2766234388 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc442ffd640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4440a7180 0x7fc444140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc4437fe640 1 -- 172.21.15.3:0/2766234388 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc444144cb0 msgr2=0x7fc4441cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc4437fe640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc444144cb0 0x7fc4441cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc4437fe640 1 -- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4440a7180 msgr2=0x7fc444140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc4437fe640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4440a7180 0x7fc444140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc4437fe640 1 -- 172.21.15.3:0/2766234388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4441ce0c0 con 0x7fc4440a6d10 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc443fff640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc444144cb0 0x7fc4441cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc442ffd640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4440a7180 0x7fc444140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc4437fe640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4440a6d10 0x7fc4440a4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c500 tx=0x7fc43400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc42bfff640 1 -- 172.21.15.3:0/2766234388 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340052d0 con 0x7fc4440a6d10 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc42bfff640 1 -- 172.21.15.3:0/2766234388 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc434005470 con 0x7fc4440a6d10 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc449f69640 1 -- 172.21.15.3:0/2766234388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc444141340 con 0x7fc4440a6d10 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc42bfff640 1 -- 172.21.15.3:0/2766234388 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434012890 con 0x7fc4440a6d10 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.149+0000 7fc449f69640 1 -- 172.21.15.3:0/2766234388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44413d270 con 0x7fc4440a6d10 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.150+0000 7fc449f69640 1 -- 172.21.15.3:0/2766234388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc444069290 con 0x7fc4440a6d10 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.152+0000 7fc42bfff640 1 -- 172.21.15.3:0/2766234388 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 14) ==== 146829+0+0 (secure 0 0 0) 0x7fc434012a30 con 0x7fc4440a6d10 2024-05-21T17:42:49.156 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.152+0000 7fc42bfff640 20 mgrc handle_mgr_map mgrmap(e 14) 2024-05-21T17:42:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.153+0000 7fc42bfff640 4 mgrc handle_mgr_map Got map version 14 2024-05-21T17:42:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.153+0000 7fc42bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.153+0000 7fc42bfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.153+0000 7fc42bfff640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fc4300af0b0 0x7fc4300b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.153+0000 7fc442ffd640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fc4300af0b0 0x7fc4300b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.157 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.153+0000 7fc42bfff640 1 -- 172.21.15.3:0/2766234388 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5198+0+0 (secure 0 0 0) 0x7fc434012d20 con 0x7fc4440a6d10 2024-05-21T17:42:49.158 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.154+0000 7fc442ffd640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fc4300af0b0 0x7fc4300b1570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc438001860 tx=0x7fc438009040 comp rx=0 tx=0).ready entity=mgr.4507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:49.159 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.158+0000 7fc42bfff640 1 -- 172.21.15.3:0/2766234388 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc4340a6d90 con 0x7fc4440a6d10 2024-05-21T17:42:49.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.165+0000 7fc42bfff640 1 -- 172.21.15.3:0/2766234388 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 15) ==== 147037+0+0 (secure 0 0 0) 0x7fc4340a2920 con 0x7fc4440a6d10 2024-05-21T17:42:49.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.165+0000 7fc42bfff640 20 mgrc handle_mgr_map mgrmap(e 15) 2024-05-21T17:42:49.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.165+0000 7fc42bfff640 4 mgrc handle_mgr_map Got map version 15 2024-05-21T17:42:49.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.165+0000 7fc42bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:49.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.389+0000 7fc449f69640 1 -- 172.21.15.3:0/2766234388 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc444073c20 con 0x7fc4440a6d10 2024-05-21T17:42:49.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.391+0000 7fc42bfff640 1 -- 172.21.15.3:0/2766234388 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fc43409e2c0 con 0x7fc4440a6d10 2024-05-21T17:42:49.392 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-21T17:42:49.393 INFO:tasks.ceph.mds.c.smithi003.stderr: -1312> 2024-05-21T17:41:27.251+0000 7fcc50a27640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-05-21T17:42:49.394 INFO:tasks.ceph.mds.c.smithi003.stderr: -834> 2024-05-21T17:41:38.266+0000 7fcc4aa1b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-21T17:42:49.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.394+0000 7fc449f69640 10 mgrc shutdown 2024-05-21T17:42:49.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.394+0000 7fc449f69640 1 -- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fc4300af0b0 msgr2=0x7fc4300b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.394+0000 7fc449f69640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fc4300af0b0 0x7fc4300b1570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc438001860 tx=0x7fc438009040 comp rx=0 tx=0).stop 2024-05-21T17:42:49.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.394+0000 7fc449f69640 1 -- 172.21.15.3:0/2766234388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4440a6d10 msgr2=0x7fc4440a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.394+0000 7fc449f69640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4440a6d10 0x7fc4440a4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c500 tx=0x7fc43400c9c0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.395+0000 7fc449f69640 1 -- 172.21.15.3:0/2766234388 shutdown_connections 2024-05-21T17:42:49.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.395+0000 7fc449f69640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7fc4300af0b0 0x7fc4300b1570 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.395+0000 7fc449f69640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7fc444144cb0 0x7fc4441cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.395+0000 7fc449f69640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4440a7180 0x7fc444140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.395+0000 7fc449f69640 1 --2- 172.21.15.3:0/2766234388 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc4440a6d10 0x7fc4440a4dc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.395+0000 7fc449f69640 1 -- 172.21.15.3:0/2766234388 >> 172.21.15.3:0/2766234388 conn(0x7fc4440a8c40 msgr2=0x7fc4440b8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:49.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.395+0000 7fc449f69640 1 -- 172.21.15.3:0/2766234388 shutdown_connections 2024-05-21T17:42:49.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.395+0000 7fc449f69640 1 -- 172.21.15.3:0/2766234388 wait complete. 2024-05-21T17:42:49.408 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-21T17:42:49.420 INFO:tasks.ceph.mds.c.smithi003.stdout:starting mds.c at 2024-05-21T17:42:49.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.513+0000 7f33e3a8e640 1 Processor -- start 2024-05-21T17:42:49.514 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.513+0000 7f33e3a8e640 1 -- start start 2024-05-21T17:42:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e3a8e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc0ac330 0x7f33dc0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e3a8e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f33dc146650 0x7f33dc150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e3a8e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33dc146b20 con 0x7f33dc149720 2024-05-21T17:42:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e3a8e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f33dc0a8530 con 0x7f33dc146650 2024-05-21T17:42:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e3a8e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f33dc146de0 con 0x7f33dc0ac330 2024-05-21T17:42:49.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e182a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc0ac330 0x7f33dc0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e182a640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc0ac330 0x7f33dc0ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51674/0 (socket says 172.21.15.3:51674) 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e182a640 1 -- 172.21.15.3:0/1772863022 learned_addr learned my addr 172.21.15.3:0/1772863022 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e202b640 1 --2- 172.21.15.3:0/1772863022 >> v2:172.21.15.203:3300/0 conn(0x7f33dc146650 0x7f33dc150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e182a640 1 -- 172.21.15.3:0/1772863022 >> v2:172.21.15.203:3300/0 conn(0x7f33dc146650 msgr2=0x7f33dc150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e182a640 1 --2- 172.21.15.3:0/1772863022 >> v2:172.21.15.203:3300/0 conn(0x7f33dc146650 0x7f33dc150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e182a640 1 -- 172.21.15.3:0/1772863022 >> v1:172.21.15.203:6789/0 conn(0x7f33dc149720 legacy=0x7f33dc149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.514+0000 7f33e182a640 1 -- 172.21.15.3:0/1772863022 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33dc0ac710 con 0x7f33dc0ac330 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.515+0000 7f33e182a640 1 --2- 172.21.15.3:0/1772863022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc0ac330 0x7f33dc0ad480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009870 tx=0x7f33d802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e949218ddb66dd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.515+0000 7f33e0828640 1 -- 172.21.15.3:0/1772863022 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d80317c0 con 0x7f33dc0ac330 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.515+0000 7f33e0828640 1 -- 172.21.15.3:0/1772863022 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f33d8037070 con 0x7f33dc0ac330 2024-05-21T17:42:49.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.515+0000 7f33e0828640 1 -- 172.21.15.3:0/1772863022 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8031ba0 con 0x7f33dc0ac330 2024-05-21T17:42:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.516+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1772863022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc0ac330 msgr2=0x7f33dc0ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.516+0000 7f33e3a8e640 1 --2- 172.21.15.3:0/1772863022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc0ac330 0x7f33dc0ad480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009870 tx=0x7f33d802b860 comp rx=0 tx=0).stop 2024-05-21T17:42:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.516+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1772863022 shutdown_connections 2024-05-21T17:42:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.516+0000 7f33e3a8e640 1 --2- 172.21.15.3:0/1772863022 >> v2:172.21.15.203:3300/0 conn(0x7f33dc146650 0x7f33dc150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.516+0000 7f33e3a8e640 1 --2- 172.21.15.3:0/1772863022 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc0ac330 0x7f33dc0ad480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.516+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1772863022 >> 172.21.15.3:0/1772863022 conn(0x7f33dc0aab00 msgr2=0x7f33dc0aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:49.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.516+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1772863022 shutdown_connections 2024-05-21T17:42:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.516+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1772863022 wait complete. 2024-05-21T17:42:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.517+0000 7f33e3a8e640 1 Processor -- start 2024-05-21T17:42:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.517+0000 7f33e3a8e640 1 -- start start 2024-05-21T17:42:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.517+0000 7f33e3a8e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33dc0ac330 0x7f33dc1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.517+0000 7f33e3a8e640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc146650 0x7f33dc13bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.517+0000 7f33e3a8e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33dc149720 0x7f33dc137200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.517+0000 7f33e3a8e640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc06ce70 con 0x7f33dc149720 2024-05-21T17:42:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.517+0000 7f33e3a8e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc06ccf0 con 0x7f33dc0ac330 2024-05-21T17:42:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.517+0000 7f33e3a8e640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f33dc14a210 con 0x7f33dc146650 2024-05-21T17:42:49.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e1029640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc146650 0x7f33dc13bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e202b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33dc149720 0x7f33dc137200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e1029640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc146650 0x7f33dc13bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51686/0 (socket says 172.21.15.3:51686) 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e1029640 1 -- 172.21.15.3:0/1314024743 learned_addr learned my addr 172.21.15.3:0/1314024743 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e182a640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33dc0ac330 0x7f33dc1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e1029640 1 -- 172.21.15.3:0/1314024743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33dc0ac330 msgr2=0x7f33dc1401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e1029640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33dc0ac330 0x7f33dc1401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e1029640 1 -- 172.21.15.3:0/1314024743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33dc149720 msgr2=0x7f33dc137200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e1029640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33dc149720 0x7f33dc137200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e1029640 1 -- 172.21.15.3:0/1314024743 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33dc13c380 con 0x7f33dc146650 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e202b640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33dc149720 0x7f33dc137200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e1029640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc146650 0x7f33dc13bb10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00b420 tx=0x7f33cc00b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:49.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33d27fc640 1 -- 172.21.15.3:0/1314024743 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc011830 con 0x7f33dc146650 2024-05-21T17:42:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33d27fc640 1 -- 172.21.15.3:0/1314024743 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f33cc016070 con 0x7f33dc146650 2024-05-21T17:42:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33d27fc640 1 -- 172.21.15.3:0/1314024743 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc011c30 con 0x7f33dc146650 2024-05-21T17:42:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1314024743 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33dc13c640 con 0x7f33dc146650 2024-05-21T17:42:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.518+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1314024743 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33dc137740 con 0x7f33dc146650 2024-05-21T17:42:49.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.519+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1314024743 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33dc069a90 con 0x7f33dc146650 2024-05-21T17:42:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.521+0000 7f33d27fc640 1 -- 172.21.15.3:0/1314024743 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 15) ==== 147037+0+0 (secure 0 0 0) 0x7f33cc004020 con 0x7f33dc146650 2024-05-21T17:42:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.521+0000 7f33d27fc640 20 mgrc handle_mgr_map mgrmap(e 15) 2024-05-21T17:42:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.521+0000 7f33d27fc640 4 mgrc handle_mgr_map Got map version 15 2024-05-21T17:42:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.521+0000 7f33d27fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.521+0000 7f33d27fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.521+0000 7f33d27fc640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f33c00af270 0x7f33c00b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.521+0000 7f33d27fc640 1 -- 172.21.15.3:0/1314024743 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 5284+0+0 (secure 0 0 0) 0x7f33cc0dcf50 con 0x7f33dc146650 2024-05-21T17:42:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.522+0000 7f33e182a640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f33c00af270 0x7f33c00b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:49.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.522+0000 7f33e182a640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f33c00af270 0x7f33c00b1730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f33d8036000 tx=0x7f33d803d040 comp rx=0 tx=0).ready entity=mgr.4507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:49.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.524+0000 7f33d27fc640 1 -- 172.21.15.3:0/1314024743 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f33cc020020 con 0x7f33dc146650 2024-05-21T17:42:49.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:49.761+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1314024743 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f33dc06d5c0 con 0x7f33dc146650 2024-05-21T17:42:50.396 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.395+0000 7f33d27fc640 1 -- 172.21.15.3:0/1314024743 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f33cc0a6950 con 0x7f33dc146650 2024-05-21T17:42:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 10 mgrc shutdown 2024-05-21T17:42:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1314024743 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f33c00af270 msgr2=0x7f33c00b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:50.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f33c00af270 0x7f33c00b1730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f33d8036000 tx=0x7f33d803d040 comp rx=0 tx=0).stop 2024-05-21T17:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1314024743 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc146650 msgr2=0x7f33dc13bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc146650 0x7f33dc13bb10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00b420 tx=0x7f33cc00b8e0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1314024743 shutdown_connections 2024-05-21T17:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f33c00af270 0x7f33c00b1730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f33dc149720 0x7f33dc137200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f33dc146650 0x7f33dc13bb10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 --2- 172.21.15.3:0/1314024743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f33dc0ac330 0x7f33dc1401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1314024743 >> 172.21.15.3:0/1314024743 conn(0x7f33dc0aab00 msgr2=0x7f33dc0bc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1314024743 shutdown_connections 2024-05-21T17:42:50.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.398+0000 7f33e3a8e640 1 -- 172.21.15.3:0/1314024743 wait complete. 2024-05-21T17:42:50.413 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-21T17:42:50.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.568+0000 7f3fc2cd4640 1 Processor -- start 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.568+0000 7f3fc2cd4640 1 -- start start 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.568+0000 7f3fc2cd4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3fbc074750 0x7f3fbc146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.568+0000 7f3fc2cd4640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc147020 0x7f3fbc14e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.568+0000 7f3fc2cd4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fbc0a3150 con 0x7f3fbc0a1720 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.568+0000 7f3fc2cd4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7f3fbc0a2070 con 0x7f3fbc074750 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.568+0000 7f3fc2cd4640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f3fbc1453d0 con 0x7f3fbc147020 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.568+0000 7f3fc1271640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc147020 0x7f3fbc14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.569+0000 7f3fc1271640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc147020 0x7f3fbc14e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51696/0 (socket says 172.21.15.3:51696) 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.569+0000 7f3fc1271640 1 -- 172.21.15.3:0/783635747 learned_addr learned my addr 172.21.15.3:0/783635747 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.569+0000 7f3fbbfff640 1 --2- 172.21.15.3:0/783635747 >> v2:172.21.15.203:3300/0 conn(0x7f3fbc074750 0x7f3fbc146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:50.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.569+0000 7f3fc1271640 1 -- 172.21.15.3:0/783635747 >> v2:172.21.15.203:3300/0 conn(0x7f3fbc074750 msgr2=0x7f3fbc146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:50.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.569+0000 7f3fc1271640 1 --2- 172.21.15.3:0/783635747 >> v2:172.21.15.203:3300/0 conn(0x7f3fbc074750 0x7f3fbc146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.569+0000 7f3fc1271640 1 -- 172.21.15.3:0/783635747 >> v1:172.21.15.203:6789/0 conn(0x7f3fbc0a1720 legacy=0x7f3fbc0a2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:50.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.569+0000 7f3fc1271640 1 -- 172.21.15.3:0/783635747 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc06ead0 con 0x7f3fbc147020 2024-05-21T17:42:50.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.569+0000 7f3fbbfff640 1 --2- 172.21.15.3:0/783635747 >> v2:172.21.15.203:3300/0 conn(0x7f3fbc074750 0x7f3fbc146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T17:42:50.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fc1271640 1 --2- 172.21.15.3:0/783635747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc147020 0x7f3fbc14e800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009e50 tx=0x7f3fb402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add6626974572e80 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:50.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fbb7fe640 1 -- 172.21.15.3:0/783635747 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb402ca70 con 0x7f3fbc147020 2024-05-21T17:42:50.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fbb7fe640 1 -- 172.21.15.3:0/783635747 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3fb402cc10 con 0x7f3fbc147020 2024-05-21T17:42:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fbb7fe640 1 -- 172.21.15.3:0/783635747 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40343f0 con 0x7f3fbc147020 2024-05-21T17:42:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/783635747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc147020 msgr2=0x7f3fbc14e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fc2cd4640 1 --2- 172.21.15.3:0/783635747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc147020 0x7f3fbc14e800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009e50 tx=0x7f3fb402bed0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/783635747 shutdown_connections 2024-05-21T17:42:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fc2cd4640 1 --2- 172.21.15.3:0/783635747 >> v2:172.21.15.203:3300/0 conn(0x7f3fbc074750 0x7f3fbc146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fc2cd4640 1 --2- 172.21.15.3:0/783635747 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc147020 0x7f3fbc14e800 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/783635747 >> 172.21.15.3:0/783635747 conn(0x7f3fbc06ccf0 msgr2=0x7f3fbc0a6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/783635747 shutdown_connections 2024-05-21T17:42:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.570+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/783635747 wait complete. 2024-05-21T17:42:50.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.571+0000 7f3fc2cd4640 1 Processor -- start 2024-05-21T17:42:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.571+0000 7f3fc2cd4640 1 -- start start 2024-05-21T17:42:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.572+0000 7f3fc2cd4640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc074750 0x7f3fbc13f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.572+0000 7f3fc2cd4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fbc0a1720 0x7f3fbc139fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.572+0000 7f3fc2cd4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc147020 0x7f3fbc1dcd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.572+0000 7f3fc2cd4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc06e050 con 0x7f3fbc0a1720 2024-05-21T17:42:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.572+0000 7f3fc2cd4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc06da10 con 0x7f3fbc147020 2024-05-21T17:42:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.572+0000 7f3fc2cd4640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7f3fbc1492a0 con 0x7f3fbc074750 2024-05-21T17:42:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.572+0000 7f3fc0a70640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc074750 0x7f3fbc13f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.572+0000 7f3fbbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fbc0a1720 0x7f3fbc139fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:50.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.572+0000 7f3fc0a70640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc074750 0x7f3fbc13f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51700/0 (socket says 172.21.15.3:51700) 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.572+0000 7f3fc0a70640 1 -- 172.21.15.3:0/1635095574 learned_addr learned my addr 172.21.15.3:0/1635095574 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fc1271640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc147020 0x7f3fbc1dcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fc0a70640 1 -- 172.21.15.3:0/1635095574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc147020 msgr2=0x7f3fbc1dcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fc0a70640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc147020 0x7f3fbc1dcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fc0a70640 1 -- 172.21.15.3:0/1635095574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fbc0a1720 msgr2=0x7f3fbc139fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fc0a70640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fbc0a1720 0x7f3fbc139fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fc0a70640 1 -- 172.21.15.3:0/1635095574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc1dd840 con 0x7f3fbc074750 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fc0a70640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc074750 0x7f3fbc13f1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c480 tx=0x7f3fac00c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fb97fa640 1 -- 172.21.15.3:0/1635095574 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac0052d0 con 0x7f3fbc074750 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fb97fa640 1 -- 172.21.15.3:0/1635095574 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3fac005470 con 0x7f3fbc074750 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fb97fa640 1 -- 172.21.15.3:0/1635095574 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac012890 con 0x7f3fbc074750 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/1635095574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fbc1ddae0 con 0x7f3fbc074750 2024-05-21T17:42:50.577 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.573+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/1635095574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fbc148d40 con 0x7f3fbc074750 2024-05-21T17:42:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.574+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/1635095574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fbc148fd0 con 0x7f3fbc074750 2024-05-21T17:42:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.575+0000 7f3fb97fa640 1 -- 172.21.15.3:0/1635095574 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 15) ==== 147037+0+0 (secure 0 0 0) 0x7f3fac012a30 con 0x7f3fbc074750 2024-05-21T17:42:50.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.575+0000 7f3fb97fa640 20 mgrc handle_mgr_map mgrmap(e 15) 2024-05-21T17:42:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.576+0000 7f3fb97fa640 4 mgrc handle_mgr_map Got map version 15 2024-05-21T17:42:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.576+0000 7f3fb97fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.576+0000 7f3fb97fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.576+0000 7f3fb97fa640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f3fa80af2f0 0x7f3fa80b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.577+0000 7f3fb97fa640 1 -- 172.21.15.3:0/1635095574 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 5284+0+0 (secure 0 0 0) 0x7f3fac0dd3d0 con 0x7f3fbc074750 2024-05-21T17:42:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.577+0000 7f3fbbfff640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f3fa80af2f0 0x7f3fa80b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:50.579 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.577+0000 7f3fbbfff640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f3fa80af2f0 0x7f3fa80b17b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0b5ff0 tx=0x7f3fb0008040 comp rx=0 tx=0).ready entity=mgr.4507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:50.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.580+0000 7f3fb97fa640 1 -- 172.21.15.3:0/1635095574 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3fac0a6dd0 con 0x7f3fbc074750 2024-05-21T17:42:50.771 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:50.770+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/1635095574 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3fbc0a3500 con 0x7f3fbc074750 2024-05-21T17:42:51.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.411+0000 7f3fb97fa640 1 -- 172.21.15.3:0/1635095574 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mgrmap(e 16) ==== 147093+0+0 (secure 0 0 0) 0x7f3fac0a2960 con 0x7f3fbc074750 2024-05-21T17:42:51.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.411+0000 7f3fb97fa640 20 mgrc handle_mgr_map mgrmap(e 16) 2024-05-21T17:42:51.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.412+0000 7f3fb97fa640 4 mgrc handle_mgr_map Got map version 16 2024-05-21T17:42:51.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.412+0000 7f3fb97fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:51.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.412+0000 7f3fb97fa640 1 -- 172.21.15.3:0/1635095574 <== mon.2 v2:172.21.15.3:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f3fac0df660 con 0x7f3fbc074750 2024-05-21T17:42:51.413 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-05-21T17:42:51.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.414+0000 7f3fc2cd4640 10 mgrc shutdown 2024-05-21T17:42:51.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.414+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/1635095574 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f3fa80af2f0 msgr2=0x7f3fa80b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:51.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.414+0000 7f3fc2cd4640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f3fa80af2f0 0x7f3fa80b17b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0b5ff0 tx=0x7f3fb0008040 comp rx=0 tx=0).stop 2024-05-21T17:42:51.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.414+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/1635095574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc074750 msgr2=0x7f3fbc13f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:51.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.414+0000 7f3fc2cd4640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc074750 0x7f3fbc13f1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c480 tx=0x7f3fac00c940 comp rx=0 tx=0).stop 2024-05-21T17:42:51.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.415+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/1635095574 shutdown_connections 2024-05-21T17:42:51.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.415+0000 7f3fc2cd4640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7f3fa80af2f0 0x7f3fa80b17b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:51.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.415+0000 7f3fc2cd4640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fbc147020 0x7f3fbc1dcd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:51.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.415+0000 7f3fc2cd4640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fbc0a1720 0x7f3fbc139fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:51.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.415+0000 7f3fc2cd4640 1 --2- 172.21.15.3:0/1635095574 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7f3fbc074750 0x7f3fbc13f1e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:51.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.415+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/1635095574 >> 172.21.15.3:0/1635095574 conn(0x7f3fbc06ccf0 msgr2=0x7f3fbc0a5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:51.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.415+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/1635095574 shutdown_connections 2024-05-21T17:42:51.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.415+0000 7f3fc2cd4640 1 -- 172.21.15.3:0/1635095574 wait complete. 2024-05-21T17:42:51.429 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-21T17:42:51.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.541+0000 7efd29139640 1 Processor -- start 2024-05-21T17:42:51.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.541+0000 7efd29139640 1 -- start start 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd29139640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd240ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd29139640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efd241387f0 0x7efd2406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd29139640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd24138c80 con 0x7efd240ad6d0 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd29139640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 -- 0x7efd24139810 con 0x7efd241387f0 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd29139640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7efd24138f20 con 0x7efd240abf70 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd22d9d640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd240ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd22d9d640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd240ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51714/0 (socket says 172.21.15.3:51714) 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd22d9d640 1 -- 172.21.15.3:0/155053987 learned_addr learned my addr 172.21.15.3:0/155053987 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd2259c640 1 --2- 172.21.15.3:0/155053987 >> v2:172.21.15.203:3300/0 conn(0x7efd241387f0 0x7efd2406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd22d9d640 1 -- 172.21.15.3:0/155053987 >> v2:172.21.15.203:3300/0 conn(0x7efd241387f0 msgr2=0x7efd2406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:51.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd22d9d640 1 --2- 172.21.15.3:0/155053987 >> v2:172.21.15.203:3300/0 conn(0x7efd241387f0 0x7efd2406c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd22d9d640 1 -- 172.21.15.3:0/155053987 >> v1:172.21.15.203:6789/0 conn(0x7efd240ad6d0 legacy=0x7efd241497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd22d9d640 1 -- 172.21.15.3:0/155053987 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2413c970 con 0x7efd240abf70 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.542+0000 7efd2259c640 1 --2- 172.21.15.3:0/155053987 >> v2:172.21.15.203:3300/0 conn(0x7efd241387f0 0x7efd2406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.543+0000 7efd22d9d640 1 --2- 172.21.15.3:0/155053987 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd240ac350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efd14009870 tx=0x7efd1402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=884729084e934ba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.543+0000 7efd21d9b640 1 -- 172.21.15.3:0/155053987 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd140317c0 con 0x7efd240abf70 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.543+0000 7efd21d9b640 1 -- 172.21.15.3:0/155053987 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efd14037070 con 0x7efd240abf70 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.543+0000 7efd21d9b640 1 -- 172.21.15.3:0/155053987 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14031ba0 con 0x7efd240abf70 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.543+0000 7efd29139640 1 -- 172.21.15.3:0/155053987 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 msgr2=0x7efd240ac350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.543+0000 7efd29139640 1 --2- 172.21.15.3:0/155053987 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd240ac350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efd14009870 tx=0x7efd1402b860 comp rx=0 tx=0).stop 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.544+0000 7efd29139640 1 -- 172.21.15.3:0/155053987 shutdown_connections 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.544+0000 7efd29139640 1 --2- 172.21.15.3:0/155053987 >> v2:172.21.15.203:3300/0 conn(0x7efd241387f0 0x7efd2406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.544+0000 7efd29139640 1 --2- 172.21.15.3:0/155053987 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd240ac350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:51.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.544+0000 7efd29139640 1 -- 172.21.15.3:0/155053987 >> 172.21.15.3:0/155053987 conn(0x7efd240aab70 msgr2=0x7efd240aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:51.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.544+0000 7efd29139640 1 -- 172.21.15.3:0/155053987 shutdown_connections 2024-05-21T17:42:51.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.544+0000 7efd29139640 1 -- 172.21.15.3:0/155053987 wait complete. 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.545+0000 7efd29139640 1 Processor -- start 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.545+0000 7efd29139640 1 -- start start 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.545+0000 7efd29139640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd24134890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.545+0000 7efd29139640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd240ad6d0 0x7efd240a8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.545+0000 7efd29139640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd241341c0 0x7efd24132030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.545+0000 7efd29139640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efd240a5250 con 0x7efd241341c0 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.545+0000 7efd29139640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efd240a50d0 con 0x7efd240ad6d0 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.545+0000 7efd29139640 1 -- --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_getmap magic: 0 -- 0x7efd240adb00 con 0x7efd240abf70 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd22d9d640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd24134890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd2359e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd241341c0 0x7efd24132030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:51.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd22d9d640 1 --2- >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd24134890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3301/0 says I am v2:172.21.15.3:51728/0 (socket says 172.21.15.3:51728) 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd22d9d640 1 -- 172.21.15.3:0/2108593387 learned_addr learned my addr 172.21.15.3:0/2108593387 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd2259c640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd240ad6d0 0x7efd240a8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd22d9d640 1 -- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd240ad6d0 msgr2=0x7efd240a8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd22d9d640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd240ad6d0 0x7efd240a8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd22d9d640 1 -- 172.21.15.3:0/2108593387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd241341c0 msgr2=0x7efd24132030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd22d9d640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd241341c0 0x7efd24132030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd22d9d640 1 -- 172.21.15.3:0/2108593387 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd24132780 con 0x7efd240abf70 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd2259c640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd240ad6d0 0x7efd240a8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd2359e640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd241341c0 0x7efd24132030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd22d9d640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd24134890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd1402b8a0 tx=0x7efd140046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd0b7fe640 1 -- 172.21.15.3:0/2108593387 <== mon.2 v2:172.21.15.3:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14032860 con 0x7efd240abf70 2024-05-21T17:42:51.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd0b7fe640 1 -- 172.21.15.3:0/2108593387 <== mon.2 v2:172.21.15.3:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efd14032a20 con 0x7efd240abf70 2024-05-21T17:42:51.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd29139640 1 -- 172.21.15.3:0/2108593387 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd24132a40 con 0x7efd240abf70 2024-05-21T17:42:51.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd0b7fe640 1 -- 172.21.15.3:0/2108593387 <== mon.2 v2:172.21.15.3:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14032e30 con 0x7efd240abf70 2024-05-21T17:42:51.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.546+0000 7efd29139640 1 -- 172.21.15.3:0/2108593387 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd240a71f0 con 0x7efd240abf70 2024-05-21T17:42:51.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.547+0000 7efd29139640 1 -- 172.21.15.3:0/2108593387 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd2413bf80 con 0x7efd240abf70 2024-05-21T17:42:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.549+0000 7efd0b7fe640 1 -- 172.21.15.3:0/2108593387 <== mon.2 v2:172.21.15.3:3301/0 4 ==== mgrmap(e 16) ==== 147093+0+0 (secure 0 0 0) 0x7efd140453f0 con 0x7efd240abf70 2024-05-21T17:42:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.549+0000 7efd0b7fe640 20 mgrc handle_mgr_map mgrmap(e 16) 2024-05-21T17:42:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.549+0000 7efd0b7fe640 4 mgrc handle_mgr_map Got map version 16 2024-05-21T17:42:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.549+0000 7efd0b7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.549+0000 7efd0b7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] 2024-05-21T17:42:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.549+0000 7efd0b7fe640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7efd100af380 0x7efd100b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-21T17:42:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.550+0000 7efd0b7fe640 1 -- 172.21.15.3:0/2108593387 <== mon.2 v2:172.21.15.3:3301/0 5 ==== osd_map(29..29 src has 1..29) ==== 4846+0+0 (secure 0 0 0) 0x7efd140ff390 con 0x7efd240abf70 2024-05-21T17:42:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.550+0000 7efd2259c640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7efd100af380 0x7efd100b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-21T17:42:51.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.550+0000 7efd2259c640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7efd100af380 0x7efd100b1840 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd240a7760 tx=0x7efd18008040 comp rx=0 tx=0).ready entity=mgr.4507 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-21T17:42:51.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.552+0000 7efd0b7fe640 1 -- 172.21.15.3:0/2108593387 <== mon.2 v2:172.21.15.3:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efd14053080 con 0x7efd240abf70 2024-05-21T17:42:51.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:51.742+0000 7efd29139640 1 -- 172.21.15.3:0/2108593387 --> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efd2406d540 con 0x7efd240abf70 2024-05-21T17:42:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.412+0000 7efd0b7fe640 1 -- 172.21.15.3:0/2108593387 <== mon.2 v2:172.21.15.3:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7efd14040070 con 0x7efd240abf70 2024-05-21T17:42:52.414 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-05-21T17:42:52.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.415+0000 7efd29139640 10 mgrc shutdown 2024-05-21T17:42:52.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 -- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7efd100af380 msgr2=0x7efd100b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:52.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7efd100af380 0x7efd100b1840 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd240a7760 tx=0x7efd18008040 comp rx=0 tx=0).stop 2024-05-21T17:42:52.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 -- 172.21.15.3:0/2108593387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 msgr2=0x7efd24134890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-21T17:42:52.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd24134890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd1402b8a0 tx=0x7efd140046c0 comp rx=0 tx=0).stop 2024-05-21T17:42:52.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 -- 172.21.15.3:0/2108593387 shutdown_connections 2024-05-21T17:42:52.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:6824/2970980678,v1:172.21.15.203:6825/2970980678] conn(0x7efd100af380 0x7efd100b1840 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd240a7760 tx=0x7efd18008040 comp rx=0 tx=0).stop 2024-05-21T17:42:52.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efd241341c0 0x7efd24132030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:52.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd240ad6d0 0x7efd240a8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:52.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 --2- 172.21.15.3:0/2108593387 >> [v2:172.21.15.3:3301/0,v1:172.21.15.3:6790/0] conn(0x7efd240abf70 0x7efd24134890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-21T17:42:52.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 -- 172.21.15.3:0/2108593387 >> 172.21.15.3:0/2108593387 conn(0x7efd240aab70 msgr2=0x7efd240aa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-21T17:42:52.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 -- 172.21.15.3:0/2108593387 shutdown_connections 2024-05-21T17:42:52.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-21T17:42:52.416+0000 7efd29139640 1 -- 172.21.15.3:0/2108593387 wait complete. 2024-05-21T17:42:52.432 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-21T17:42:52.432 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-21T17:42:52.432 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.442 INFO:tasks.ceph.mds.a:Stopped 2024-05-21T17:42:52.442 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-21T17:42:52.442 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.492 INFO:tasks.ceph.mds.c:Stopped 2024-05-21T17:42:52.492 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-21T17:42:52.492 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.503 INFO:tasks.ceph.mds.b:Stopped 2024-05-21T17:42:52.503 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-21T17:42:52.504 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-21T17:42:52.504 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.565 INFO:tasks.ceph.osd.0:Stopped 2024-05-21T17:42:52.565 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-21T17:42:52.565 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.602 INFO:tasks.ceph.osd.1:Stopped 2024-05-21T17:42:52.602 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-21T17:42:52.602 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.673 INFO:tasks.ceph.osd.2:Stopped 2024-05-21T17:42:52.673 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-21T17:42:52.673 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.750 INFO:tasks.ceph.osd.3:Stopped 2024-05-21T17:42:52.750 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-21T17:42:52.750 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-21T17:42:52.750 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.779 INFO:tasks.ceph.mgr.x:Stopped 2024-05-21T17:42:52.779 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-21T17:42:52.779 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.818 INFO:tasks.ceph.mgr.y:Stopped 2024-05-21T17:42:52.818 DEBUG:tasks.ceph.mgr.z:waiting for process to exit 2024-05-21T17:42:52.818 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.888 INFO:tasks.ceph.mgr.z:Stopped 2024-05-21T17:42:52.888 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-21T17:42:52.888 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-21T17:42:52.889 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.903 INFO:tasks.ceph.mon.a:Stopped 2024-05-21T17:42:52.903 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-21T17:42:52.903 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.957 INFO:tasks.ceph.mon.c:Stopped 2024-05-21T17:42:52.957 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-21T17:42:52.957 INFO:teuthology.orchestra.run:waiting for 300 2024-05-21T17:42:52.969 INFO:tasks.ceph.mon.b:Stopped 2024-05-21T17:42:52.969 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-21T17:42:52.969 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v 'replacing it with standby' | egrep -v 'No standby daemons available' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-05-21T17:42:53.031 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi003.front.sepia.ceph.com 2024-05-21T17:42:53.031 DEBUG:teuthology.orchestra.run.smithi003:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-21T17:42:53.539 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi203.front.sepia.ceph.com 2024-05-21T17:42:53.540 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-21T17:42:53.961 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi203.front.sepia.ceph.com 2024-05-21T17:42:53.961 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-21T17:42:54.025 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi203.front.sepia.ceph.com 2024-05-21T17:42:54.026 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-21T17:42:54.064 INFO:tasks.ceph:Archiving mon data... 2024-05-21T17:42:54.066 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528/data/mon.a.tgz 2024-05-21T17:42:54.066 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-05-21T17:42:54.085 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.vvFgKEzwUS 2024-05-21T17:42:54.086 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.vvFgKEzwUS 2024-05-21T17:42:54.351 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.vvFgKEzwUS 2024-05-21T17:42:54.446 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.vvFgKEzwUS is 957KB 2024-05-21T17:42:54.531 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.vvFgKEzwUS 2024-05-21T17:42:54.547 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528/data/mon.c.tgz 2024-05-21T17:42:54.547 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-05-21T17:42:54.602 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.9DVEVBMdnW 2024-05-21T17:42:54.602 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.9DVEVBMdnW 2024-05-21T17:42:54.886 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.9DVEVBMdnW 2024-05-21T17:42:54.971 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.9DVEVBMdnW is 1MB 2024-05-21T17:42:55.052 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.9DVEVBMdnW 2024-05-21T17:42:55.067 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528/data/mon.b.tgz 2024-05-21T17:42:55.067 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-05-21T17:42:55.086 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.Dj6ePnbENo 2024-05-21T17:42:55.086 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Dj6ePnbENo 2024-05-21T17:42:55.374 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.Dj6ePnbENo 2024-05-21T17:42:55.456 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.Dj6ePnbENo is 1MB 2024-05-21T17:42:55.564 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.Dj6ePnbENo 2024-05-21T17:42:55.579 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-21T17:42:55.580 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-21T17:42:55.582 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-21T17:42:55.664 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-21T17:42:55.699 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-21T17:42:55.699 INFO:tasks.ceph:Archiving crash dumps... 2024-05-21T17:42:55.701 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528/remote/smithi003/crash 2024-05-21T17:42:55.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-21T17:42:55.730 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528/remote/smithi203/crash 2024-05-21T17:42:55.731 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-21T17:42:55.764 INFO:tasks.ceph:Shutting down logrotate 2024-05-21T17:42:55.764 DEBUG:teuthology.orchestra.run.smithi003:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-21T17:42:55.775 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-21T17:42:55.853 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-21T17:42:55.878 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-21T17:42:55.903 INFO:tasks.ceph:Compressing logs... 2024-05-21T17:42:55.903 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T17:42:55.921 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-21T17:42:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28542.log 2024-05-21T17:42:55.944 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30363.log 2024-05-21T17:42:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30153.log 2024-05-21T17:42:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32083.log 2024-05-21T17:42:55.970 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28542.log.gz 2024-05-21T17:42:55.970 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30363.log.gz 2024-05-21T17:42:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30111.log 2024-05-21T17:42:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31685.log 2024-05-21T17:42:55.970 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30153.log.gz 2024-05-21T17:42:55.970 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.32083.log: /var/log/ceph/ceph-client.admin.30111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29652.log 2024-05-21T17:42:55.970 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32083.log.gz 2024-05-21T17:42:55.970 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30111.log.gz 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28311.log 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29652.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31685.log.gz -5 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30803.log 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28311.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29652.log.gz 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30844.log 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30803.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28311.log.gz 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.30803.log.gz 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31749.log 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30844.log.gz -5 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.29842.log 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30672.log 2024-05-21T17:42:55.971 INFO:teuthology.orchestra.run.smithi003.stderr: 90.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31749.log.gz 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31630.log 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29842.log: /var/log/ceph/ceph-client.admin.30672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30675.log 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29842.log.gz 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31630.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30672.log.gz 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31999.log 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31630.log.gz 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32228.log 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30675.log.gz 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31999.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.32228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29512.log 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31999.log.gz 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.audit.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32228.log.gz 2024-05-21T17:42:55.972 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30504.log 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30507.log 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30504.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29512.log.gzgzip 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30504.log.gz 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30065.log 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30507.log.gz 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30506.log 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30846.log 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30506.log: gzip 87.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.30065.log.gz 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.32041.log 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30506.log.gz 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31077.log 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.32041.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30846.log.gz 2024-05-21T17:42:55.973 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31035.log 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32041.log.gz 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29800.log 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31077.log.gz 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31035.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29800.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31035.log.gz -5 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.29800.log.gz 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32185.log 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.32185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31822.log 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32185.log.gz 2024-05-21T17:42:55.974 INFO:teuthology.orchestra.run.smithi003.stderr: 90.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30279.log 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31822.log: /var/log/ceph/ceph-client.admin.30279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31957.log 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31822.log.gz 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30279.log.gz 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31120.log 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29554.log 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31957.log: /var/log/ceph/ceph-client.admin.31120.log: gzip -5 --verbose 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31957.log.gz 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.32125.log 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29554.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31120.log.gz 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29454.log 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29554.log.gz 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.32125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30673.log 2024-05-21T17:42:55.975 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29454.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32125.log.gz 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31523.log 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.29454.log.gz 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30405.log 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30673.log.gz 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:gzip/var/log/ceph/ceph-client.admin.31523.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30951.log 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30405.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31523.log.gz 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26164.log 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30405.log.gz 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28495.log 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph.tmp-client.admin.26164.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26164.log.gz 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.30951.log.gz 2024-05-21T17:42:55.976 INFO:teuthology.orchestra.run.smithi003.stderr: -- /var/log/ceph/ceph-client.admin.31162.log 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.28495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30505.log 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28495.log.gz 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30783.log 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31162.log.gz 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30237.log 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5/var/log/ceph/ceph-client.admin.30783.log: --verbose -- /var/log/ceph/ceph-client.admin.31774.log 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30505.log.gz 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30237.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30783.log.gz 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30321.log 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30237.log.gz 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32270.log 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31774.log.gz 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31204.log 2024-05-21T17:42:55.977 INFO:teuthology.orchestra.run.smithi003.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30321.log.gz 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.32270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31846.log 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:gzip/var/log/ceph/ceph-client.admin.31204.log: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32270.log.gz 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30195.log 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31846.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31204.log.gz 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30993.log 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31846.log.gz 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30195.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31660.log 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30993.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30195.log.gz 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30674.log 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.30993.log.gz 2024-05-21T17:42:55.978 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30023.log 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29886.log 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31660.log.gz 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.30674.log: /var/log/ceph/ceph-client.admin.30023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29930.log 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30674.log.gz 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31246.log 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30023.log.gz 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31798.log 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29886.log.gz 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31547.log 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31246.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29930.log.gz 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31571.log 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29981.log 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31798.log.gz 2024-05-21T17:42:55.979 INFO:teuthology.orchestra.run.smithi003.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.31246.log.gz 2024-05-21T17:42:55.980 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.31571.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31547.log.gz -5 2024-05-21T17:42:55.980 INFO:teuthology.orchestra.run.smithi003.stderr: --verbose -- /var/log/ceph/ceph-client.admin.29758.log 2024-05-21T17:42:55.980 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29981.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31571.log.gz 2024-05-21T17:42:55.980 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.29758.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29981.log.gz 2024-05-21T17:42:55.980 INFO:teuthology.orchestra.run.smithi003.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29758.log.gz 2024-05-21T17:42:55.984 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-21T17:42:55.984 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-21T17:42:55.984 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31004.log 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr: 83.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.31004.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.audit.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31004.log.gz 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30964.log 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr: 88.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.3.log: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30924.log 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.30964.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.30964.log.gz 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.30924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30884.log 2024-05-21T17:42:55.985 INFO:teuthology.orchestra.run.smithi203.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.30924.log.gz 2024-05-21T17:42:55.986 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.z.log 2024-05-21T17:42:55.986 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.30884.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-21T17:42:55.986 INFO:teuthology.orchestra.run.smithi203.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.30884.log.gz 2024-05-21T17:42:55.986 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mgr.z.log: /var/log/ceph/ceph-mon.b.log: 91.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-21T17:42:55.997 INFO:teuthology.orchestra.run.smithi003.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-21T17:42:56.003 INFO:teuthology.orchestra.run.smithi203.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mgr.z.log.gz 2024-05-21T17:42:56.057 INFO:teuthology.orchestra.run.smithi203.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-21T17:42:56.057 INFO:teuthology.orchestra.run.smithi003.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-21T17:42:56.614 INFO:teuthology.orchestra.run.smithi203.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-21T17:42:56.647 INFO:teuthology.orchestra.run.smithi203.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-21T17:42:56.677 INFO:teuthology.orchestra.run.smithi203.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-21T17:42:56.679 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-05-21T17:42:56.679 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m0.720s 2024-05-21T17:42:56.679 INFO:teuthology.orchestra.run.smithi203.stderr:user 0m2.092s 2024-05-21T17:42:56.679 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m0.091s 2024-05-21T17:42:56.696 INFO:teuthology.orchestra.run.smithi003.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-21T17:42:56.698 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-21T17:42:56.698 INFO:teuthology.orchestra.run.smithi003.stderr:real 0m0.764s 2024-05-21T17:42:56.699 INFO:teuthology.orchestra.run.smithi003.stderr:user 0m0.915s 2024-05-21T17:42:56.699 INFO:teuthology.orchestra.run.smithi003.stderr:sys 0m0.078s 2024-05-21T17:42:56.699 INFO:tasks.ceph:Archiving logs... 2024-05-21T17:42:56.699 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528/remote/smithi003/log 2024-05-21T17:42:56.700 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T17:42:56.944 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528/remote/smithi203/log 2024-05-21T17:42:56.944 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-21T17:42:57.251 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-21T17:42:57.260 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-21T17:42:57.260 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T17:42:57.263 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-21T17:42:57.342 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-21T17:42:57.351 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-21T17:42:57.351 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T17:42:57.353 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-21T17:42:57.366 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-05-21T17:42:57.368 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-05-21T17:42:57.370 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T17:42:57.370 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-05-21T17:42:57.371 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv01.front.sepia.ceph.com 3 6 177 19 -501us[ -504us] +/- 46ms 2024-05-21T17:42:57.371 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv02.front.sepia.ceph.com 3 6 177 20 -604us[ -608us] +/- 59ms 2024-05-21T17:42:57.371 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 19 +1535us[+1535us] +/- 73ms 2024-05-21T17:42:57.371 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T17:42:57.372 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-21T17:42:57.372 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-05-21T17:42:57.372 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv01.front.sepia.ceph.com 3 6 177 19 -493us[ -493us] +/- 46ms 2024-05-21T17:42:57.372 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv02.front.sepia.ceph.com 3 6 177 19 -598us[ -598us] +/- 59ms 2024-05-21T17:42:57.372 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 20 +1535us[+1535us] +/- 72ms 2024-05-21T17:42:57.373 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-21T17:42:57.373 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-21T17:42:57.390 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-21T17:42:57.391 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-21T17:42:57.399 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T17:42:57.475 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-21T17:42:57.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T17:42:57.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T17:42:57.563 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-21T17:42:57.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T17:42:57.590 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-21T17:42:57.600 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-21T17:42:57.608 INFO:teuthology.task.internal:Duration was 481.701262 seconds 2024-05-21T17:42:57.608 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-21T17:42:57.617 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-21T17:42:57.617 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T17:42:57.620 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-21T17:42:57.657 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T17:42:57.668 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-21T17:42:57.940 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-21T17:42:57.940 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-05-21T17:42:57.940 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T17:42:57.962 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-05-21T17:42:57.962 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-21T17:42:58.006 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-21T17:42:58.006 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T17:42:58.009 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-21T17:42:58.072 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-21T17:42:58.072 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T17:42:58.076 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-21T17:42:58.205 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-21T17:42:58.215 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-21T17:42:58.215 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T17:42:58.240 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-21T17:42:58.265 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-21T17:42:58.274 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T17:42:58.283 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-21T17:42:58.305 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-05-21T17:42:58.328 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-05-21T17:42:58.350 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T17:42:58.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T17:42:58.377 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-21T17:42:58.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-21T17:42:58.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-21T17:42:58.413 INFO:teuthology.task.internal:Transferring archived files... 2024-05-21T17:42:58.414 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528/remote/smithi003 2024-05-21T17:42:58.414 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T17:42:58.449 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-05-21_01:07:55-rados-wip-yuri7-testing-2024-05-20-1227-distro-default-smithi/7717528/remote/smithi203 2024-05-21T17:42:58.449 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-21T17:42:58.481 INFO:teuthology.task.internal:Removing archive directory... 2024-05-21T17:42:58.482 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T17:42:58.486 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-21T17:42:58.531 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-21T17:42:58.540 INFO:teuthology.task.internal:Not uploading archives. 2024-05-21T17:42:58.540 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-21T17:42:58.550 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-21T17:42:58.550 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T17:42:58.552 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-21T17:42:58.565 INFO:teuthology.orchestra.run.smithi003.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 17:42 /home/ubuntu/cephtest 2024-05-21T17:42:58.587 INFO:teuthology.orchestra.run.smithi203.stdout: 265296 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 21 17:42 /home/ubuntu/cephtest 2024-05-21T17:42:58.588 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-21T17:42:58.597 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-21T17:42:58.617 INFO:teuthology.run:Summary data: description: rados/mgr/{clusters/{2-node-mgr} debug/mgr distro/{centos_latest} mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/{1-install 2-ceph 3-mgrmodules 4-units/workunits}} duration: 481.70126247406006 flavor: default owner: scheduled_yuriw@teuthology success: true 2024-05-21T17:42:58.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-21T17:42:58.700 INFO:teuthology.run:pass