2024-09-16T14:54:21.360 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T14:54:21.376 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T14:54:21.469 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795 branch: reef description: rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/workunits} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906795' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 2 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef ceph: conf: client: debug client: 20 debug mgrc: 20 debug ms: 1 global: mon election default strategy: 1 mds: debug mgrc: 20 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mgr ttl cache expire seconds: 0 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug mgrc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 owner: scheduled_teuthology@teuthology priority: 931 repo: https://git.ceph.com/ceph.git roles: - - mgr.x - mon.a - mon.c - mds.a - mds.c - osd.0 - client.0 - - mgr.y - mgr.z - mon.b - mds.b - osd.1 - osd.2 - osd.3 - client.1 seed: 1390 sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 sleep_before_teardown: 0 subset: 5319/100000 suite: rados suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 targets: smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ0hLxykiCI0jl6iX32iCl4ULylUNnJXigJVoSgYcwXwuyBuQh4A/AJ76JEgMXRdCWEYR45us25VmrYCffVPdwE= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNckmitjRQ9MZ/bZKvbOthOOBHXpJb9piMQfQSWEhqEA3dvN2/MeIK+QQUyj/snI7jbBkHJb/jmULZ4+sH4RFOc= tasks: - install: null - ceph: log-ignorelist: - overall HEALTH_ - \(MGR_DOWN\) - \(PG_ - replacing it with standby - No standby daemons available - \(POOL_APP_NOT_ENABLED\) wait-for-scrub: false - workunit: clients: client.0: - mgr/test_localpool.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_22:00:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T14:54:21.469 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa; will attempt to use it 2024-09-16T14:54:21.470 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa/tasks 2024-09-16T14:54:21.470 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T14:54:21.471 INFO:teuthology.task.internal:Checking packages... 2024-09-16T14:54:21.497 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8' 2024-09-16T14:54:21.497 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T14:54:21.497 INFO:teuthology.packaging:ref: None 2024-09-16T14:54:21.497 INFO:teuthology.packaging:tag: None 2024-09-16T14:54:21.497 INFO:teuthology.packaging:branch: reef 2024-09-16T14:54:21.497 INFO:teuthology.packaging:sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:54:21.498 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-09-16T14:54:21.770 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-853-gae7f3733-1jammy 2024-09-16T14:54:21.772 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T14:54:21.779 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T14:54:21.779 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T14:54:21.786 INFO:teuthology.task.internal:Saving configuration 2024-09-16T14:54:21.798 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T14:54:21.805 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T14:54:21.830 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 14:48:47.455828', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK7ZHsvLoy37mEg/dGFsc85Fk85ZLPajtE6UGWGIHKy4'} 2024-09-16T14:54:21.850 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 14:48:47.454348', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINgNSah0lfT7IbJiTvA6RKyrxaiK8zUtL4muZVqVBC2A'} 2024-09-16T14:54:21.850 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T14:54:21.856 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'] 2024-09-16T14:54:21.857 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1'] 2024-09-16T14:54:21.857 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T14:54:21.918 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6d9817f1c0>, signals=[15]) 2024-09-16T14:54:21.918 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T14:54:21.926 INFO:teuthology.task.internal:Opening connections... 2024-09-16T14:54:21.926 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-09-16T14:54:21.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:54:22.013 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-09-16T14:54:22.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:54:22.094 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T14:54:22.100 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-09-16T14:54:22.105 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-09-16T14:54:22.105 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-09-16T14:54:22.151 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Ubuntu" 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="22.04" 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_CODENAME=jammy 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:ID=ubuntu 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE=debian 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T14:54:22.152 INFO:teuthology.orchestra.run.smithi133.stdout:UBUNTU_CODENAME=jammy 2024-09-16T14:54:22.153 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-09-16T14:54:22.175 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-09-16T14:54:22.181 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-09-16T14:54:22.181 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-09-16T14:54:22.228 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-16T14:54:22.228 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Ubuntu" 2024-09-16T14:54:22.228 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="22.04" 2024-09-16T14:54:22.228 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-16T14:54:22.229 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_CODENAME=jammy 2024-09-16T14:54:22.229 INFO:teuthology.orchestra.run.smithi191.stdout:ID=ubuntu 2024-09-16T14:54:22.229 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE=debian 2024-09-16T14:54:22.229 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-16T14:54:22.229 INFO:teuthology.orchestra.run.smithi191.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-16T14:54:22.229 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-16T14:54:22.229 INFO:teuthology.orchestra.run.smithi191.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-16T14:54:22.229 INFO:teuthology.orchestra.run.smithi191.stdout:UBUNTU_CODENAME=jammy 2024-09-16T14:54:22.229 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-09-16T14:54:22.250 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T14:54:22.258 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T14:54:22.266 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T14:54:22.267 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-09-16T14:54:22.269 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-09-16T14:54:22.275 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T14:54:22.280 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T14:54:22.280 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-09-16T14:54:22.317 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-09-16T14:54:22.336 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T14:54:22.391 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T14:54:22.405 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T14:54:22.405 INFO:teuthology.task.kernel:config {'mgr.x': {'kdb': 1, 'sha1': 'distro'}, 'mgr.y': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T14:54:22.405 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T14:54:22.406 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T14:54:22.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:54:22.410 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-09-16T14:54:22.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:54:22.411 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-09-16T14:54:22.456 INFO:teuthology.orchestra.run.smithi133.stdout:5.15.0-56-generic 2024-09-16T14:54:22.456 INFO:teuthology.task.kernel:Running kernel on smithi133: 5.15.0-56-generic 2024-09-16T14:54:22.456 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get clean 2024-09-16T14:54:22.457 INFO:teuthology.orchestra.run.smithi191.stdout:5.15.0-56-generic 2024-09-16T14:54:22.457 INFO:teuthology.task.kernel:Running kernel on smithi191: 5.15.0-56-generic 2024-09-16T14:54:22.457 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get clean 2024-09-16T14:54:22.579 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get update 2024-09-16T14:54:22.607 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2024-09-16T14:54:22.851 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T14:54:22.860 INFO:teuthology.orchestra.run.smithi191.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-16T14:54:22.909 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T14:54:22.910 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-16T14:54:22.916 INFO:teuthology.orchestra.run.smithi191.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-16T14:54:22.952 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-16T14:54:23.269 INFO:teuthology.orchestra.run.smithi191.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-16T14:54:23.290 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-16T14:54:23.307 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-16T14:54:23.329 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-16T14:54:23.409 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,025 kB] 2024-09-16T14:54:23.446 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,025 kB] 2024-09-16T14:54:23.582 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-16T14:54:23.726 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-16T14:54:23.755 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-16T14:54:23.761 INFO:teuthology.orchestra.run.smithi191.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [353 kB] 2024-09-16T14:54:23.763 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-16T14:54:23.764 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-16T14:54:23.768 INFO:teuthology.orchestra.run.smithi191.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-16T14:54:23.769 INFO:teuthology.orchestra.run.smithi191.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-16T14:54:23.770 INFO:teuthology.orchestra.run.smithi191.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-16T14:54:23.776 INFO:teuthology.orchestra.run.smithi191.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-16T14:54:23.777 INFO:teuthology.orchestra.run.smithi191.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,441 kB] 2024-09-16T14:54:23.780 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-16T14:54:23.789 INFO:teuthology.orchestra.run.smithi133.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-16T14:54:23.789 INFO:teuthology.orchestra.run.smithi133.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-16T14:54:23.798 INFO:teuthology.orchestra.run.smithi133.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [353 kB] 2024-09-16T14:54:23.813 INFO:teuthology.orchestra.run.smithi133.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-16T14:54:23.814 INFO:teuthology.orchestra.run.smithi133.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,441 kB] 2024-09-16T14:54:23.847 INFO:teuthology.orchestra.run.smithi191.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-16T14:54:23.847 INFO:teuthology.orchestra.run.smithi191.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [420 kB] 2024-09-16T14:54:23.881 INFO:teuthology.orchestra.run.smithi133.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-16T14:54:23.882 INFO:teuthology.orchestra.run.smithi133.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [420 kB] 2024-09-16T14:54:23.886 INFO:teuthology.orchestra.run.smithi133.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-16T14:54:23.887 INFO:teuthology.orchestra.run.smithi133.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-16T14:54:23.917 INFO:teuthology.orchestra.run.smithi191.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-16T14:54:23.917 INFO:teuthology.orchestra.run.smithi191.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-16T14:54:23.917 INFO:teuthology.orchestra.run.smithi191.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-16T14:54:23.924 INFO:teuthology.orchestra.run.smithi191.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-16T14:54:23.925 INFO:teuthology.orchestra.run.smithi191.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-16T14:54:23.932 INFO:teuthology.orchestra.run.smithi191.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-16T14:54:23.940 INFO:teuthology.orchestra.run.smithi191.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-16T14:54:23.951 INFO:teuthology.orchestra.run.smithi191.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-16T14:54:23.951 INFO:teuthology.orchestra.run.smithi191.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-16T14:54:23.953 INFO:teuthology.orchestra.run.smithi191.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-16T14:54:23.954 INFO:teuthology.orchestra.run.smithi191.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-16T14:54:23.955 INFO:teuthology.orchestra.run.smithi191.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-16T14:54:23.955 INFO:teuthology.orchestra.run.smithi191.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-16T14:54:23.956 INFO:teuthology.orchestra.run.smithi191.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-16T14:54:23.957 INFO:teuthology.orchestra.run.smithi133.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-16T14:54:23.957 INFO:teuthology.orchestra.run.smithi133.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-16T14:54:23.957 INFO:teuthology.orchestra.run.smithi133.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-16T14:54:23.958 INFO:teuthology.orchestra.run.smithi133.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-16T14:54:23.958 INFO:teuthology.orchestra.run.smithi191.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-16T14:54:23.958 INFO:teuthology.orchestra.run.smithi191.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-16T14:54:23.971 INFO:teuthology.orchestra.run.smithi191.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-16T14:54:23.972 INFO:teuthology.orchestra.run.smithi191.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-16T14:54:23.973 INFO:teuthology.orchestra.run.smithi133.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-16T14:54:23.977 INFO:teuthology.orchestra.run.smithi133.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-16T14:54:23.984 INFO:teuthology.orchestra.run.smithi133.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-16T14:54:23.987 INFO:teuthology.orchestra.run.smithi133.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-16T14:54:23.988 INFO:teuthology.orchestra.run.smithi133.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-16T14:54:23.989 INFO:teuthology.orchestra.run.smithi133.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-16T14:54:23.990 INFO:teuthology.orchestra.run.smithi133.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-16T14:54:23.995 INFO:teuthology.orchestra.run.smithi133.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-16T14:54:23.996 INFO:teuthology.orchestra.run.smithi133.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-16T14:54:23.996 INFO:teuthology.orchestra.run.smithi133.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-16T14:54:24.001 INFO:teuthology.orchestra.run.smithi191.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-16T14:54:24.008 INFO:teuthology.orchestra.run.smithi133.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-16T14:54:24.009 INFO:teuthology.orchestra.run.smithi133.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-16T14:54:24.025 INFO:teuthology.orchestra.run.smithi133.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-16T14:54:24.026 INFO:teuthology.orchestra.run.smithi133.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-16T14:54:24.040 INFO:teuthology.orchestra.run.smithi191.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-16T14:54:24.042 INFO:teuthology.orchestra.run.smithi191.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-16T14:54:24.043 INFO:teuthology.orchestra.run.smithi191.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-16T14:54:24.044 INFO:teuthology.orchestra.run.smithi191.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-16T14:54:24.044 INFO:teuthology.orchestra.run.smithi191.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-16T14:54:24.053 INFO:teuthology.orchestra.run.smithi191.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-16T14:54:24.054 INFO:teuthology.orchestra.run.smithi191.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-16T14:54:24.055 INFO:teuthology.orchestra.run.smithi191.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-16T14:54:24.078 INFO:teuthology.orchestra.run.smithi133.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-16T14:54:24.080 INFO:teuthology.orchestra.run.smithi133.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-16T14:54:24.081 INFO:teuthology.orchestra.run.smithi133.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-16T14:54:24.082 INFO:teuthology.orchestra.run.smithi133.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-16T14:54:24.082 INFO:teuthology.orchestra.run.smithi133.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-16T14:54:24.089 INFO:teuthology.orchestra.run.smithi133.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-16T14:54:24.090 INFO:teuthology.orchestra.run.smithi133.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-16T14:54:24.091 INFO:teuthology.orchestra.run.smithi133.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-16T14:54:30.059 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 16.0 MB in 3s (5,791 kB/s) 2024-09-16T14:54:30.076 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 16.0 MB in 3s (5,680 kB/s) 2024-09-16T14:54:31.491 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-16T14:54:31.503 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-16T14:54:31.510 DEBUG:teuthology.orchestra.run.smithi191:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T14:54:31.521 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T14:54:31.571 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-16T14:54:31.583 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-16T14:54:31.780 INFO:teuthology.orchestra.run.smithi191.stdout:Building dependency tree... 2024-09-16T14:54:31.781 INFO:teuthology.orchestra.run.smithi191.stdout:Reading state information... 2024-09-16T14:54:31.793 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2024-09-16T14:54:31.794 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2024-09-16T14:54:31.954 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T14:54:31.954 INFO:teuthology.orchestra.run.smithi191.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T14:54:31.954 INFO:teuthology.orchestra.run.smithi191.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T14:54:31.955 INFO:teuthology.orchestra.run.smithi191.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T14:54:31.955 INFO:teuthology.orchestra.run.smithi191.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-16T14:54:31.955 INFO:teuthology.orchestra.run.smithi191.stdout: python2.7-minimal 2024-09-16T14:54:31.955 INFO:teuthology.orchestra.run.smithi191.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T14:54:31.956 INFO:teuthology.orchestra.run.smithi191.stdout:The following additional packages will be installed: 2024-09-16T14:54:31.956 INFO:teuthology.orchestra.run.smithi191.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T14:54:31.956 INFO:teuthology.orchestra.run.smithi191.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-16T14:54:31.956 INFO:teuthology.orchestra.run.smithi191.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-16T14:54:31.958 INFO:teuthology.orchestra.run.smithi191.stdout:Suggested packages: 2024-09-16T14:54:31.958 INFO:teuthology.orchestra.run.smithi191.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-16T14:54:31.958 INFO:teuthology.orchestra.run.smithi191.stdout:Recommended packages: 2024-09-16T14:54:31.958 INFO:teuthology.orchestra.run.smithi191.stdout: thermald 2024-09-16T14:54:31.981 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T14:54:31.981 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T14:54:31.981 INFO:teuthology.orchestra.run.smithi133.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T14:54:31.982 INFO:teuthology.orchestra.run.smithi133.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T14:54:31.982 INFO:teuthology.orchestra.run.smithi133.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-16T14:54:31.982 INFO:teuthology.orchestra.run.smithi133.stdout: python2.7-minimal 2024-09-16T14:54:31.982 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T14:54:31.983 INFO:teuthology.orchestra.run.smithi133.stdout:The following additional packages will be installed: 2024-09-16T14:54:31.983 INFO:teuthology.orchestra.run.smithi133.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T14:54:31.983 INFO:teuthology.orchestra.run.smithi133.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-16T14:54:31.984 INFO:teuthology.orchestra.run.smithi133.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-16T14:54:31.985 INFO:teuthology.orchestra.run.smithi133.stdout:Suggested packages: 2024-09-16T14:54:31.985 INFO:teuthology.orchestra.run.smithi133.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-16T14:54:31.985 INFO:teuthology.orchestra.run.smithi133.stdout:Recommended packages: 2024-09-16T14:54:31.985 INFO:teuthology.orchestra.run.smithi133.stdout: thermald 2024-09-16T14:54:31.997 INFO:teuthology.orchestra.run.smithi191.stdout:The following NEW packages will be installed: 2024-09-16T14:54:31.997 INFO:teuthology.orchestra.run.smithi191.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T14:54:31.997 INFO:teuthology.orchestra.run.smithi191.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-16T14:54:31.997 INFO:teuthology.orchestra.run.smithi191.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-16T14:54:31.998 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages will be upgraded: 2024-09-16T14:54:31.999 INFO:teuthology.orchestra.run.smithi191.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T14:54:32.033 INFO:teuthology.orchestra.run.smithi133.stdout:The following NEW packages will be installed: 2024-09-16T14:54:32.033 INFO:teuthology.orchestra.run.smithi133.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-16T14:54:32.033 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-16T14:54:32.034 INFO:teuthology.orchestra.run.smithi133.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-16T14:54:32.035 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages will be upgraded: 2024-09-16T14:54:32.035 INFO:teuthology.orchestra.run.smithi133.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-16T14:54:32.086 INFO:teuthology.orchestra.run.smithi133.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T14:54:32.086 INFO:teuthology.orchestra.run.smithi133.stdout:Need to get 113 MB of archives. 2024-09-16T14:54:32.087 INFO:teuthology.orchestra.run.smithi133.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-16T14:54:32.087 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-16T14:54:32.178 INFO:teuthology.orchestra.run.smithi191.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T14:54:32.178 INFO:teuthology.orchestra.run.smithi191.stdout:Need to get 113 MB of archives. 2024-09-16T14:54:32.178 INFO:teuthology.orchestra.run.smithi191.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-16T14:54:32.178 INFO:teuthology.orchestra.run.smithi191.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-16T14:54:32.447 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-16T14:54:32.537 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-16T14:54:33.310 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-16T14:54:33.310 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-16T14:54:33.310 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-16T14:54:33.428 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-16T14:54:33.525 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-16T14:54:33.695 INFO:teuthology.orchestra.run.smithi191.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-16T14:54:33.922 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 113 MB in 1s (76.4 MB/s) 2024-09-16T14:54:34.046 INFO:teuthology.orchestra.run.smithi191.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-16T14:54:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-16T14:54:36.153 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-16T14:54:36.157 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T14:54:36.275 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:36.278 INFO:teuthology.orchestra.run.smithi191.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-16T14:54:36.278 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-16T14:54:36.279 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-16T14:54:36.626 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-16T14:54:36.707 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-16T14:54:37.098 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 113 MB in 5s (24.1 MB/s) 2024-09-16T14:54:37.270 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-16T14:54:38.751 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-16T14:54:38.773 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T14:54:38.892 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:39.344 INFO:teuthology.orchestra.run.smithi191.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-16T14:54:39.348 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T14:54:39.416 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:39.704 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-16T14:54:39.725 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T14:54:39.770 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:41.767 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-16T14:54:41.788 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T14:54:41.879 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:42.601 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-16T14:54:42.622 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T14:54:42.728 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:46.387 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T14:54:46.537 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T14:54:46.885 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T14:54:47.023 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T14:54:47.343 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-16T14:54:47.366 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-16T14:54:47.409 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T14:54:49.181 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T14:54:49.306 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T14:54:49.615 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T14:54:49.735 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T14:54:50.048 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-16T14:54:50.070 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-16T14:54:50.114 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T14:54:53.147 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-16T14:54:53.184 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T14:54:53.214 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:55.983 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-16T14:54:56.020 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-16T14:54:56.058 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:56.584 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T14:54:56.733 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T14:54:57.107 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T14:54:57.254 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:57.356 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-16T14:54:57.458 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:58.943 INFO:teuthology.orchestra.run.smithi133.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-16T14:54:58.943 INFO:teuthology.orchestra.run.smithi133.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-16T14:54:58.943 INFO:teuthology.orchestra.run.smithi133.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-16T14:54:58.943 INFO:teuthology.orchestra.run.smithi133.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-16T14:54:59.278 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:54:59.690 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-16T14:54:59.804 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-16T14:55:00.176 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-16T14:55:00.301 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:55:00.427 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-16T14:55:00.555 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:55:00.712 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:55:01.893 INFO:teuthology.orchestra.run.smithi191.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-16T14:55:01.894 INFO:teuthology.orchestra.run.smithi191.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-16T14:55:01.894 INFO:teuthology.orchestra.run.smithi191.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-16T14:55:01.894 INFO:teuthology.orchestra.run.smithi191.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-16T14:55:02.058 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:55:02.130 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-16T14:55:02.273 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-16T14:55:02.382 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:55:02.449 INFO:teuthology.orchestra.run.smithi133.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T14:55:02.450 INFO:teuthology.orchestra.run.smithi133.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-16T14:55:03.451 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:55:05.228 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-16T14:55:05.354 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-16T14:55:05.480 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-16T14:55:05.547 INFO:teuthology.orchestra.run.smithi191.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-16T14:55:05.547 INFO:teuthology.orchestra.run.smithi191.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-16T14:55:16.790 INFO:teuthology.orchestra.run.smithi133.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T14:55:16.791 INFO:teuthology.orchestra.run.smithi133.stdout:Sourcing file `/etc/default/grub' 2024-09-16T14:55:16.817 INFO:teuthology.orchestra.run.smithi133.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T14:55:16.845 INFO:teuthology.orchestra.run.smithi133.stdout:Generating grub configuration file ... 2024-09-16T14:55:17.188 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T14:55:17.214 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T14:55:17.566 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T14:55:17.572 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T14:55:17.643 INFO:teuthology.orchestra.run.smithi133.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T14:55:17.648 INFO:teuthology.orchestra.run.smithi133.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T14:55:17.821 INFO:teuthology.orchestra.run.smithi133.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T14:55:17.821 INFO:teuthology.orchestra.run.smithi133.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T14:55:17.821 INFO:teuthology.orchestra.run.smithi133.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T14:55:17.863 INFO:teuthology.orchestra.run.smithi133.stdout:done 2024-09-16T14:55:18.326 DEBUG:teuthology.orchestra.run.smithi133:> dpkg -s linux-image-generic 2024-09-16T14:55:18.347 INFO:teuthology.orchestra.run.smithi133.stdout:Package: linux-image-generic 2024-09-16T14:55:18.348 INFO:teuthology.orchestra.run.smithi133.stdout:Status: install ok installed 2024-09-16T14:55:18.348 INFO:teuthology.orchestra.run.smithi133.stdout:Priority: optional 2024-09-16T14:55:18.348 INFO:teuthology.orchestra.run.smithi133.stdout:Section: kernel 2024-09-16T14:55:18.348 INFO:teuthology.orchestra.run.smithi133.stdout:Installed-Size: 21 2024-09-16T14:55:18.348 INFO:teuthology.orchestra.run.smithi133.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T14:55:18.348 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: amd64 2024-09-16T14:55:18.348 INFO:teuthology.orchestra.run.smithi133.stdout:Source: linux-meta 2024-09-16T14:55:18.348 INFO:teuthology.orchestra.run.smithi133.stdout:Version: 5.15.0.121.121 2024-09-16T14:55:18.348 INFO:teuthology.orchestra.run.smithi133.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T14:55:18.349 INFO:teuthology.orchestra.run.smithi133.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T14:55:18.349 INFO:teuthology.orchestra.run.smithi133.stdout:Recommends: thermald 2024-09-16T14:55:18.349 INFO:teuthology.orchestra.run.smithi133.stdout:Description: Generic Linux kernel image 2024-09-16T14:55:18.349 INFO:teuthology.orchestra.run.smithi133.stdout: This package will always depend on the latest generic kernel image 2024-09-16T14:55:18.349 INFO:teuthology.orchestra.run.smithi133.stdout: available. 2024-09-16T14:55:18.349 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-16T14:55:18.349 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T14:55:18.349 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T14:55:18.349 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-16T14:55:18.350 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T14:55:18.350 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get clean 2024-09-16T14:55:18.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2024-09-16T14:55:18.689 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T14:55:18.697 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T14:55:18.714 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T14:55:18.810 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T14:55:19.441 INFO:teuthology.orchestra.run.smithi191.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-16T14:55:19.441 INFO:teuthology.orchestra.run.smithi191.stdout:Sourcing file `/etc/default/grub' 2024-09-16T14:55:19.453 INFO:teuthology.orchestra.run.smithi191.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T14:55:19.490 INFO:teuthology.orchestra.run.smithi191.stdout:Generating grub configuration file ... 2024-09-16T14:55:19.729 INFO:teuthology.orchestra.run.smithi191.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T14:55:19.745 INFO:teuthology.orchestra.run.smithi191.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T14:55:20.118 INFO:teuthology.orchestra.run.smithi191.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T14:55:20.123 INFO:teuthology.orchestra.run.smithi191.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T14:55:20.164 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-16T14:55:20.183 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T14:55:20.193 INFO:teuthology.orchestra.run.smithi191.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T14:55:20.199 INFO:teuthology.orchestra.run.smithi191.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T14:55:20.237 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-16T14:55:20.381 INFO:teuthology.orchestra.run.smithi191.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T14:55:20.381 INFO:teuthology.orchestra.run.smithi191.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T14:55:20.381 INFO:teuthology.orchestra.run.smithi191.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T14:55:20.421 INFO:teuthology.orchestra.run.smithi191.stdout:done 2024-09-16T14:55:20.423 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2024-09-16T14:55:20.424 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2024-09-16T14:55:20.590 INFO:teuthology.orchestra.run.smithi133.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-16T14:55:20.590 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T14:55:20.590 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T14:55:20.591 INFO:teuthology.orchestra.run.smithi133.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T14:55:20.591 INFO:teuthology.orchestra.run.smithi133.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T14:55:20.591 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T14:55:20.591 INFO:teuthology.orchestra.run.smithi133.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T14:55:20.591 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T14:55:20.631 INFO:teuthology.orchestra.run.smithi133.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T14:55:20.633 DEBUG:teuthology.orchestra.run.smithi133:> dpkg -s linux-image-generic 2024-09-16T14:55:20.689 INFO:teuthology.orchestra.run.smithi133.stdout:Package: linux-image-generic 2024-09-16T14:55:20.689 INFO:teuthology.orchestra.run.smithi133.stdout:Status: install ok installed 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Priority: optional 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Section: kernel 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Installed-Size: 21 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: amd64 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Source: linux-meta 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Version: 5.15.0.121.121 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Recommends: thermald 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout:Description: Generic Linux kernel image 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout: This package will always depend on the latest generic kernel image 2024-09-16T14:55:20.690 INFO:teuthology.orchestra.run.smithi133.stdout: available. 2024-09-16T14:55:20.691 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-16T14:55:20.735 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.78wiygAO0q 2024-09-16T14:55:20.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub/grub.cfg /tmp/tmp.78wiygAO0q 2024-09-16T14:55:20.790 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.78wiygAO0q 2024-09-16T14:55:20.911 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.78wiygAO0q is 10KB 2024-09-16T14:55:20.915 DEBUG:teuthology.orchestra.run.smithi191:> dpkg -s linux-image-generic 2024-09-16T14:55:20.923 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.78wiygAO0q 2024-09-16T14:55:20.929 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Package: linux-image-generic 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Status: install ok installed 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Priority: optional 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Section: kernel 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Installed-Size: 21 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: amd64 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Source: linux-meta 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Version: 5.15.0.121.121 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T14:55:20.935 INFO:teuthology.orchestra.run.smithi191.stdout:Recommends: thermald 2024-09-16T14:55:20.936 INFO:teuthology.orchestra.run.smithi191.stdout:Description: Generic Linux kernel image 2024-09-16T14:55:20.936 INFO:teuthology.orchestra.run.smithi191.stdout: This package will always depend on the latest generic kernel image 2024-09-16T14:55:20.936 INFO:teuthology.orchestra.run.smithi191.stdout: available. 2024-09-16T14:55:20.936 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-16T14:55:20.936 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T14:55:20.936 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T14:55:20.936 INFO:teuthology.task.kernel:Installing distro kernel on mgr.y... 2024-09-16T14:55:20.936 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi191.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T14:55:20.936 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get clean 2024-09-16T14:55:20.981 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T14:55:20.982 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T14:55:20.982 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T14:55:21.048 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-16T14:55:21.048 DEBUG:teuthology.orchestra.run.smithi133:> sudo update-grub 2024-09-16T14:55:21.104 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get update 2024-09-16T14:55:21.359 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T14:55:21.430 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T14:55:21.431 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T14:55:21.511 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T14:55:22.464 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub' 2024-09-16T14:55:22.466 INFO:teuthology.orchestra.run.smithi133.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T14:55:22.474 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-09-16T14:55:22.689 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T14:55:22.695 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T14:55:22.806 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-16T14:55:22.824 DEBUG:teuthology.orchestra.run.smithi191:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-16T14:55:22.889 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-16T14:55:22.991 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T14:55:22.996 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T14:55:23.038 INFO:teuthology.orchestra.run.smithi133.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T14:55:23.042 INFO:teuthology.orchestra.run.smithi133.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T14:55:23.102 INFO:teuthology.orchestra.run.smithi191.stdout:Building dependency tree... 2024-09-16T14:55:23.102 INFO:teuthology.orchestra.run.smithi191.stdout:Reading state information... 2024-09-16T14:55:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T14:55:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T14:55:23.153 INFO:teuthology.orchestra.run.smithi133.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T14:55:23.172 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-09-16T14:55:23.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-09-16T14:55:23.289 INFO:teuthology.orchestra.run.smithi191.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-16T14:55:23.290 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T14:55:23.290 INFO:teuthology.orchestra.run.smithi191.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T14:55:23.290 INFO:teuthology.orchestra.run.smithi191.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T14:55:23.290 INFO:teuthology.orchestra.run.smithi191.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T14:55:23.290 INFO:teuthology.orchestra.run.smithi191.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T14:55:23.290 INFO:teuthology.orchestra.run.smithi191.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T14:55:23.290 INFO:teuthology.orchestra.run.smithi191.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T14:55:23.335 INFO:teuthology.orchestra.run.smithi191.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-16T14:55:23.337 DEBUG:teuthology.orchestra.run.smithi191:> dpkg -s linux-image-generic 2024-09-16T14:55:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Package: linux-image-generic 2024-09-16T14:55:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Status: install ok installed 2024-09-16T14:55:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Priority: optional 2024-09-16T14:55:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Section: kernel 2024-09-16T14:55:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Installed-Size: 21 2024-09-16T14:55:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Maintainer: Ubuntu Kernel Team 2024-09-16T14:55:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: amd64 2024-09-16T14:55:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Source: linux-meta 2024-09-16T14:55:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Version: 5.15.0.121.121 2024-09-16T14:55:23.353 INFO:teuthology.orchestra.run.smithi191.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-16T14:55:23.354 INFO:teuthology.orchestra.run.smithi191.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-16T14:55:23.354 INFO:teuthology.orchestra.run.smithi191.stdout:Recommends: thermald 2024-09-16T14:55:23.354 INFO:teuthology.orchestra.run.smithi191.stdout:Description: Generic Linux kernel image 2024-09-16T14:55:23.354 INFO:teuthology.orchestra.run.smithi191.stdout: This package will always depend on the latest generic kernel image 2024-09-16T14:55:23.354 INFO:teuthology.orchestra.run.smithi191.stdout: available. 2024-09-16T14:55:23.355 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-09-16T14:55:23.399 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.hnTS8nZqmv 2024-09-16T14:55:23.400 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /boot/grub/grub.cfg /tmp/tmp.hnTS8nZqmv 2024-09-16T14:55:23.453 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.hnTS8nZqmv 2024-09-16T14:55:23.568 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.hnTS8nZqmv is 10KB 2024-09-16T14:55:23.581 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.hnTS8nZqmv 2024-09-16T14:55:23.586 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-16T14:55:23.637 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T14:55:23.637 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-16T14:55:23.637 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-16T14:55:23.710 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-16T14:55:23.710 DEBUG:teuthology.orchestra.run.smithi191:> sudo update-grub 2024-09-16T14:55:24.981 INFO:teuthology.orchestra.run.smithi191.stderr:Sourcing file `/etc/default/grub' 2024-09-16T14:55:24.982 INFO:teuthology.orchestra.run.smithi191.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-16T14:55:24.984 INFO:teuthology.orchestra.run.smithi191.stderr:Generating grub configuration file ... 2024-09-16T14:55:25.185 INFO:teuthology.orchestra.run.smithi191.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-16T14:55:25.191 INFO:teuthology.orchestra.run.smithi191.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-16T14:55:25.429 INFO:teuthology.orchestra.run.smithi191.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-16T14:55:25.434 INFO:teuthology.orchestra.run.smithi191.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-16T14:55:25.478 INFO:teuthology.orchestra.run.smithi191.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-16T14:55:25.484 INFO:teuthology.orchestra.run.smithi191.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-16T14:55:25.606 INFO:teuthology.orchestra.run.smithi191.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-16T14:55:25.606 INFO:teuthology.orchestra.run.smithi191.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-16T14:55:25.606 INFO:teuthology.orchestra.run.smithi191.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-16T14:55:25.622 INFO:teuthology.orchestra.run.smithi191.stderr:done 2024-09-16T14:55:25.631 DEBUG:teuthology.orchestra.run.smithi191:> sudo shutdown -r now 2024-09-16T14:55:53.259 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-16T14:55:53.260 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-16T14:55:53.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:55:55.635 INFO:teuthology.task.kernel:Checking client mgr.y for new kernel version... 2024-09-16T14:55:55.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-16T14:55:55.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:56:11.581 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-09-16T14:56:14.145 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-09-16T14:56:20.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-16T14:56:20.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:56:23.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-16T14:56:23.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:56:23.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-09-16T14:56:26.209 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-09-16T14:56:35.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-16T14:56:35.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:56:38.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-16T14:56:38.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:56:54.081 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-09-16T14:57:09.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-16T14:57:09.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:57:09.343 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-16T14:57:09.893 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-16T14:57:09.893 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.15.0-121-generic"... 2024-09-16T14:57:09.894 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-09-16T14:57:09.941 INFO:teuthology.orchestra.run.smithi133.stdout:5.15.0-121-generic 2024-09-16T14:57:09.941 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-16T14:57:09.942 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T14:57:09.942 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T14:57:10.070 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-09-16T14:57:10.903 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-16T14:57:10.903 INFO:teuthology.task.kernel:Checking kernel version of mgr.y, want "5.15.0-121-generic"... 2024-09-16T14:57:10.903 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-09-16T14:57:10.943 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-16T14:57:10.943 DEBUG:teuthology.orchestra.run.smithi133:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T14:57:10.951 INFO:teuthology.orchestra.run.smithi191.stdout:5.15.0-121-generic 2024-09-16T14:57:10.951 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-16T14:57:10.951 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T14:57:10.951 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-16T14:57:11.319 INFO:teuthology.orchestra.run.smithi133.stdout:ttyS1 2024-09-16T14:57:11.351 DEBUG:teuthology.parallel:result is None 2024-09-16T14:57:11.952 INFO:teuthology.task.kernel:Enabling kdb on mgr.y... 2024-09-16T14:57:11.952 DEBUG:teuthology.orchestra.run.smithi191:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T14:57:12.023 INFO:teuthology.orchestra.run.smithi191.stdout:ttyS1 2024-09-16T14:57:12.045 DEBUG:teuthology.parallel:result is None 2024-09-16T14:57:12.045 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T14:57:12.054 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T14:57:12.054 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T14:57:12.057 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T14:57:12.062 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T14:57:12.069 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T14:57:12.079 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T14:57:12.079 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T14:57:12.103 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T14:57:12.136 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T14:57:12.147 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T14:57:12.147 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T14:57:12.163 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T14:57:12.181 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:57:12.189 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:57:12.202 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:57:12.211 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:57:12.214 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T14:57:12.236 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T14:57:12.236 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T14:57:12.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T14:57:12.266 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T14:57:12.282 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T14:57:12.282 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T14:57:12.298 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T14:57:12.315 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:57:12.349 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:57:12.398 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T14:57:12.398 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T14:57:12.453 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:57:12.459 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:57:12.507 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T14:57:12.507 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T14:57:12.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-09-16T14:57:12.576 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-09-16T14:57:12.645 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T14:57:12.655 INFO:teuthology.task.internal:Starting timer... 2024-09-16T14:57:12.655 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T14:57:12.709 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T14:57:12.721 DEBUG:teuthology.task.selinux:Excluding smithi133: OS 'ubuntu' does not support SELinux 2024-09-16T14:57:12.721 DEBUG:teuthology.task.selinux:Excluding smithi191: OS 'ubuntu' does not support SELinux 2024-09-16T14:57:12.721 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T14:57:12.721 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-16T14:57:12.721 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T14:57:12.721 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T14:57:12.730 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T14:57:12.743 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T14:57:12.780 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi133.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T15:04:48.125 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-09-16T15:04:48.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-16T15:04:48.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:04:48.212 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-16T15:04:48.288 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-16T15:04:48.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-16T15:04:48.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:04:48.368 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-09-16T15:04:48.444 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-16T15:04:48.445 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T15:04:48.456 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T15:04:48.456 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T15:04:48.456 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:04:48.459 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T15:04:48.459 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:04:48.485 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-16T15:04:48.486 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: Command line: ntpd -gq 2024-09-16T15:04:48.486 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: ---------------------------------------------------- 2024-09-16T15:04:48.486 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: ntp-4 is maintained by Network Time Foundation, 2024-09-16T15:04:48.486 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-16T15:04:48.486 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: corporation. Support and training for ntp-4 are 2024-09-16T15:04:48.486 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: available at https://www.nwtime.org/support 2024-09-16T15:04:48.486 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: ---------------------------------------------------- 2024-09-16T15:04:48.487 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: proto: precision = 0.043 usec (-24) 2024-09-16T15:04:48.487 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: basedate set to 2022-02-04 2024-09-16T15:04:48.487 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: gps base set to 2022-02-06 (week 2196) 2024-09-16T15:04:48.487 INFO:teuthology.orchestra.run.smithi133.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T15:04:48.487 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T15:04:48.488 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T15:04:48.488 INFO:teuthology.orchestra.run.smithi133.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T15:04:48.489 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T15:04:48.489 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T15:04:48.489 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T15:04:48.489 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: Listen normally on 3 enp3s0f1 172.21.15.133:123 2024-09-16T15:04:48.489 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: Listen normally on 4 lo [::1]:123 2024-09-16T15:04:48.489 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8087%5]:123 2024-09-16T15:04:48.489 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:48 ntpd[18480]: Listening on routing socket on fd #22 for interface updates 2024-09-16T15:04:48.516 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-16T15:04:48.516 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: Command line: ntpd -gq 2024-09-16T15:04:48.516 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: ---------------------------------------------------- 2024-09-16T15:04:48.516 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: ntp-4 is maintained by Network Time Foundation, 2024-09-16T15:04:48.516 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-16T15:04:48.516 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: corporation. Support and training for ntp-4 are 2024-09-16T15:04:48.516 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: available at https://www.nwtime.org/support 2024-09-16T15:04:48.516 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: ---------------------------------------------------- 2024-09-16T15:04:48.517 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: proto: precision = 0.060 usec (-24) 2024-09-16T15:04:48.517 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: basedate set to 2022-02-04 2024-09-16T15:04:48.517 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: gps base set to 2022-02-06 (week 2196) 2024-09-16T15:04:48.517 INFO:teuthology.orchestra.run.smithi191.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T15:04:48.518 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-16T15:04:48.518 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: restrict ::: KOD does nothing without LIMITED. 2024-09-16T15:04:48.518 INFO:teuthology.orchestra.run.smithi191.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-16T15:04:48.519 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: Listen and drop on 0 v6wildcard [::]:123 2024-09-16T15:04:48.519 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-16T15:04:48.519 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: Listen normally on 2 lo 127.0.0.1:123 2024-09-16T15:04:48.519 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: Listen normally on 3 enp3s0f1 172.21.15.191:123 2024-09-16T15:04:48.519 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: Listen normally on 4 lo [::1]:123 2024-09-16T15:04:48.519 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7211%5]:123 2024-09-16T15:04:48.519 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:48 ntpd[18483]: Listening on routing socket on fd #22 for interface updates 2024-09-16T15:04:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:49.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:49.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:49.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:49.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:49.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:49 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:49.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:49.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:49 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:50.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:50 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:50.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:50 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:50.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:50 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:50.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:50 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:50.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:50 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:50.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:50 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:51.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.489 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:51.489 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:51.489 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.489 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.489 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:51.489 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:51 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:51.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:51.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:51 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:52.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:52 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:52.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:52 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:52.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:52 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:52.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:52 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:52.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:52 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:52.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:52 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:53.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:53.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:53.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:53.489 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:53 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:53.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:53.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:53.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:53.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:53.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:53.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:53.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:53.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:53.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:53.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:53.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:53.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:53 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:54.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:54 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:54.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:54 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:54.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:54 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:54.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:54 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:54.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:54 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:54.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:54 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:55.487 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:55.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:55.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:55.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:55.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:55.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:55.488 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:55.488 INFO:teuthology.orchestra.run.smithi133.stdout:16 Sep 15:04:55 ntpd[18480]: ntpd: time slew -0.000939 s 2024-09-16T15:04:55.488 INFO:teuthology.orchestra.run.smithi133.stdout:ntpd: time slew -0.000939s 2024-09-16T15:04:55.489 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T15:04:55.489 INFO:teuthology.orchestra.run.smithi133.stderr:16 Sep 15:04:55 ntpd[18480]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T15:04:55.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:55 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:55.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:55 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:55.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:55 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:55.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:55 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:55.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:55 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-16T15:04:55.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:55 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240916: Permission denied 2024-09-16T15:04:55.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:55 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-16T15:04:55.517 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:55 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240916: Permission denied 2024-09-16T15:04:55.518 INFO:teuthology.orchestra.run.smithi191.stdout:16 Sep 15:04:55 ntpd[18483]: ntpd: time slew -0.000883 s 2024-09-16T15:04:55.518 INFO:teuthology.orchestra.run.smithi191.stdout:ntpd: time slew -0.000883s 2024-09-16T15:04:55.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:55 ntpd[18483]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-16T15:04:55.518 INFO:teuthology.orchestra.run.smithi191.stderr:16 Sep 15:04:55 ntpd[18483]: can't open /var/log/ntpstats/loopstats.20240916: Permission denied 2024-09-16T15:04:55.541 INFO:teuthology.orchestra.run.smithi133.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T15:04:55.542 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================== 2024-09-16T15:04:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T15:04:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T15:04:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T15:04:55.542 INFO:teuthology.orchestra.run.smithi133.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T15:04:55.570 INFO:teuthology.orchestra.run.smithi191.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T15:04:55.571 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================== 2024-09-16T15:04:55.571 INFO:teuthology.orchestra.run.smithi191.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T15:04:55.571 INFO:teuthology.orchestra.run.smithi191.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T15:04:55.571 INFO:teuthology.orchestra.run.smithi191.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T15:04:55.571 INFO:teuthology.orchestra.run.smithi191.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T15:04:55.572 INFO:teuthology.run_tasks:Running task install... 2024-09-16T15:04:55.580 DEBUG:teuthology.task.install:project ceph 2024-09-16T15:04:55.580 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}} 2024-09-16T15:04:55.580 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'} 2024-09-16T15:04:55.580 INFO:teuthology.task.install:Using flavor: default 2024-09-16T15:04:55.585 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T15:04:55.585 INFO:teuthology.task.install:extra packages: [] 2024-09-16T15:04:55.585 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-key list | grep Ceph 2024-09-16T15:04:55.586 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-key list | grep Ceph 2024-09-16T15:04:55.641 INFO:teuthology.orchestra.run.smithi133.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-16T15:04:55.668 INFO:teuthology.orchestra.run.smithi133.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T15:04:55.668 INFO:teuthology.orchestra.run.smithi133.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T15:04:55.669 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T15:04:55.669 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T15:04:55.673 INFO:teuthology.orchestra.run.smithi191.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-16T15:04:55.704 INFO:teuthology.orchestra.run.smithi191.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-16T15:04:55.704 INFO:teuthology.orchestra.run.smithi191.stdout:uid [ unknown] Ceph.com (release key) 2024-09-16T15:04:55.704 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-16T15:04:55.704 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T15:04:55.813 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default/ 2024-09-16T15:04:55.814 INFO:teuthology.task.install.deb:Package version is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T15:04:55.836 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default/ 2024-09-16T15:04:55.836 INFO:teuthology.task.install.deb:Package version is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T15:04:55.927 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:04:55.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T15:04:55.941 DEBUG:teuthology.orchestra.run.smithi133:> sudo apt-get update 2024-09-16T15:04:55.946 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:04:55.946 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-16T15:04:55.959 DEBUG:teuthology.orchestra.run.smithi191:> sudo apt-get update 2024-09-16T15:04:56.125 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T15:04:56.125 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T15:04:56.135 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T15:04:56.152 INFO:teuthology.orchestra.run.smithi191.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T15:04:56.164 INFO:teuthology.orchestra.run.smithi133.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-16T15:04:56.180 INFO:teuthology.orchestra.run.smithi191.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy InRelease 2024-09-16T15:04:56.190 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-16T15:04:56.201 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-16T15:04:56.213 INFO:teuthology.orchestra.run.smithi133.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-16T15:04:56.224 INFO:teuthology.orchestra.run.smithi191.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-16T15:04:56.240 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-16T15:04:56.240 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-16T15:04:56.240 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-09-16T15:04:56.245 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2024-09-16T15:04:56.270 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2024-09-16T15:04:56.272 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-09-16T15:04:56.312 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-16T15:04:56.394 INFO:teuthology.orchestra.run.smithi133.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-16T15:04:56.488 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 28.8 kB in 0s (82.4 kB/s) 2024-09-16T15:04:56.621 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 28.8 kB in 0s (58.7 kB/s) 2024-09-16T15:04:57.492 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-16T15:04:57.510 DEBUG:teuthology.orchestra.run.smithi191:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-845-ge4e227b3-1jammy cephadm=18.2.4-845-ge4e227b3-1jammy ceph-mds=18.2.4-845-ge4e227b3-1jammy ceph-mgr=18.2.4-845-ge4e227b3-1jammy ceph-common=18.2.4-845-ge4e227b3-1jammy ceph-fuse=18.2.4-845-ge4e227b3-1jammy ceph-test=18.2.4-845-ge4e227b3-1jammy ceph-volume=18.2.4-845-ge4e227b3-1jammy radosgw=18.2.4-845-ge4e227b3-1jammy python3-rados=18.2.4-845-ge4e227b3-1jammy python3-rgw=18.2.4-845-ge4e227b3-1jammy python3-cephfs=18.2.4-845-ge4e227b3-1jammy python3-rbd=18.2.4-845-ge4e227b3-1jammy libcephfs2=18.2.4-845-ge4e227b3-1jammy libcephfs-dev=18.2.4-845-ge4e227b3-1jammy librados2=18.2.4-845-ge4e227b3-1jammy librbd1=18.2.4-845-ge4e227b3-1jammy rbd-fuse=18.2.4-845-ge4e227b3-1jammy 2024-09-16T15:04:57.564 INFO:teuthology.orchestra.run.smithi191.stdout:Reading package lists... 2024-09-16T15:04:57.667 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-16T15:04:57.684 DEBUG:teuthology.orchestra.run.smithi133:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-845-ge4e227b3-1jammy cephadm=18.2.4-845-ge4e227b3-1jammy ceph-mds=18.2.4-845-ge4e227b3-1jammy ceph-mgr=18.2.4-845-ge4e227b3-1jammy ceph-common=18.2.4-845-ge4e227b3-1jammy ceph-fuse=18.2.4-845-ge4e227b3-1jammy ceph-test=18.2.4-845-ge4e227b3-1jammy ceph-volume=18.2.4-845-ge4e227b3-1jammy radosgw=18.2.4-845-ge4e227b3-1jammy python3-rados=18.2.4-845-ge4e227b3-1jammy python3-rgw=18.2.4-845-ge4e227b3-1jammy python3-cephfs=18.2.4-845-ge4e227b3-1jammy python3-rbd=18.2.4-845-ge4e227b3-1jammy libcephfs2=18.2.4-845-ge4e227b3-1jammy libcephfs-dev=18.2.4-845-ge4e227b3-1jammy librados2=18.2.4-845-ge4e227b3-1jammy librbd1=18.2.4-845-ge4e227b3-1jammy rbd-fuse=18.2.4-845-ge4e227b3-1jammy 2024-09-16T15:04:57.741 INFO:teuthology.orchestra.run.smithi133.stdout:Reading package lists... 2024-09-16T15:04:57.750 INFO:teuthology.orchestra.run.smithi191.stdout:Building dependency tree... 2024-09-16T15:04:57.751 INFO:teuthology.orchestra.run.smithi191.stdout:Reading state information... 2024-09-16T15:04:57.934 INFO:teuthology.orchestra.run.smithi191.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T15:04:57.934 INFO:teuthology.orchestra.run.smithi191.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T15:04:57.934 INFO:teuthology.orchestra.run.smithi191.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T15:04:57.934 INFO:teuthology.orchestra.run.smithi191.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T15:04:57.935 INFO:teuthology.orchestra.run.smithi191.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T15:04:57.935 INFO:teuthology.orchestra.run.smithi191.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T15:04:57.935 INFO:teuthology.orchestra.run.smithi191.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T15:04:57.937 INFO:teuthology.orchestra.run.smithi133.stdout:Building dependency tree... 2024-09-16T15:04:57.937 INFO:teuthology.orchestra.run.smithi133.stdout:Reading state information... 2024-09-16T15:04:57.938 INFO:teuthology.orchestra.run.smithi191.stdout:The following additional packages will be installed: 2024-09-16T15:04:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T15:04:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-16T15:04:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-16T15:04:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-16T15:04:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-16T15:04:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-16T15:04:57.938 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T15:04:57.939 INFO:teuthology.orchestra.run.smithi191.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout:Suggested packages: 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: python3-watchdog 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout:Recommended packages: 2024-09-16T15:04:57.940 INFO:teuthology.orchestra.run.smithi191.stdout: btrfs-tools 2024-09-16T15:04:58.039 INFO:teuthology.orchestra.run.smithi191.stdout:The following NEW packages will be installed: 2024-09-16T15:04:58.039 INFO:teuthology.orchestra.run.smithi191.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T15:04:58.039 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T15:04:58.040 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-16T15:04:58.041 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-16T15:04:58.041 INFO:teuthology.orchestra.run.smithi191.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-16T15:04:58.041 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-16T15:04:58.041 INFO:teuthology.orchestra.run.smithi191.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-16T15:04:58.042 INFO:teuthology.orchestra.run.smithi191.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-16T15:04:58.042 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-16T15:04:58.042 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-16T15:04:58.042 INFO:teuthology.orchestra.run.smithi191.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-16T15:04:58.042 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-16T15:04:58.042 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-16T15:04:58.042 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-16T15:04:58.042 INFO:teuthology.orchestra.run.smithi191.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T15:04:58.042 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-16T15:04:58.043 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-16T15:04:58.043 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-16T15:04:58.043 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-16T15:04:58.043 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T15:04:58.043 INFO:teuthology.orchestra.run.smithi191.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T15:04:58.043 INFO:teuthology.orchestra.run.smithi191.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T15:04:58.043 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T15:04:58.043 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-16T15:04:58.043 INFO:teuthology.orchestra.run.smithi191.stdout: socat xmlstarlet 2024-09-16T15:04:58.101 INFO:teuthology.orchestra.run.smithi191.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-16T15:04:58.101 INFO:teuthology.orchestra.run.smithi191.stdout:Need to get 163 MB of archives. 2024-09-16T15:04:58.101 INFO:teuthology.orchestra.run.smithi191.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-16T15:04:58.101 INFO:teuthology.orchestra.run.smithi191.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-16T15:04:58.130 INFO:teuthology.orchestra.run.smithi133.stdout:The following packages were automatically installed and are no longer required: 2024-09-16T15:04:58.130 INFO:teuthology.orchestra.run.smithi133.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-16T15:04:58.130 INFO:teuthology.orchestra.run.smithi133.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-16T15:04:58.130 INFO:teuthology.orchestra.run.smithi133.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-16T15:04:58.131 INFO:teuthology.orchestra.run.smithi133.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-16T15:04:58.131 INFO:teuthology.orchestra.run.smithi133.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-16T15:04:58.131 INFO:teuthology.orchestra.run.smithi133.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-16T15:04:58.132 INFO:teuthology.orchestra.run.smithi133.stdout:The following additional packages will be installed: 2024-09-16T15:04:58.132 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-16T15:04:58.132 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-16T15:04:58.132 INFO:teuthology.orchestra.run.smithi133.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-16T15:04:58.133 INFO:teuthology.orchestra.run.smithi133.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-16T15:04:58.133 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-16T15:04:58.133 INFO:teuthology.orchestra.run.smithi133.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-16T15:04:58.133 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T15:04:58.134 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-16T15:04:58.135 INFO:teuthology.orchestra.run.smithi133.stdout:Suggested packages: 2024-09-16T15:04:58.135 INFO:teuthology.orchestra.run.smithi133.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-16T15:04:58.135 INFO:teuthology.orchestra.run.smithi133.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-16T15:04:58.135 INFO:teuthology.orchestra.run.smithi133.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-16T15:04:58.135 INFO:teuthology.orchestra.run.smithi133.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-16T15:04:58.135 INFO:teuthology.orchestra.run.smithi133.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-16T15:04:58.135 INFO:teuthology.orchestra.run.smithi133.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-16T15:04:58.135 INFO:teuthology.orchestra.run.smithi133.stdout: python3-watchdog 2024-09-16T15:04:58.135 INFO:teuthology.orchestra.run.smithi133.stdout:Recommended packages: 2024-09-16T15:04:58.136 INFO:teuthology.orchestra.run.smithi133.stdout: btrfs-tools 2024-09-16T15:04:58.160 INFO:teuthology.orchestra.run.smithi191.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-16T15:04:58.182 INFO:teuthology.orchestra.run.smithi191.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-845-ge4e227b3-1jammy [3,451 kB] 2024-09-16T15:04:58.197 INFO:teuthology.orchestra.run.smithi191.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-16T15:04:58.200 INFO:teuthology.orchestra.run.smithi191.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-16T15:04:58.215 INFO:teuthology.orchestra.run.smithi191.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-16T15:04:58.236 INFO:teuthology.orchestra.run.smithi133.stdout:The following NEW packages will be installed: 2024-09-16T15:04:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-16T15:04:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-16T15:04:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-16T15:04:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-16T15:04:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-16T15:04:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-16T15:04:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-16T15:04:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-16T15:04:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-16T15:04:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-16T15:04:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-16T15:04:58.239 INFO:teuthology.orchestra.run.smithi133.stdout: socat xmlstarlet 2024-09-16T15:04:58.273 INFO:teuthology.orchestra.run.smithi191.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-16T15:04:58.278 INFO:teuthology.orchestra.run.smithi191.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-16T15:04:58.290 INFO:teuthology.orchestra.run.smithi191.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-16T15:04:58.294 INFO:teuthology.orchestra.run.smithi191.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-16T15:04:58.295 INFO:teuthology.orchestra.run.smithi191.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-16T15:04:58.295 INFO:teuthology.orchestra.run.smithi191.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-16T15:04:58.298 INFO:teuthology.orchestra.run.smithi133.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-16T15:04:58.298 INFO:teuthology.orchestra.run.smithi133.stdout:Need to get 163 MB of archives. 2024-09-16T15:04:58.299 INFO:teuthology.orchestra.run.smithi133.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-16T15:04:58.299 INFO:teuthology.orchestra.run.smithi133.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-16T15:04:58.304 INFO:teuthology.orchestra.run.smithi191.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-16T15:04:58.304 INFO:teuthology.orchestra.run.smithi191.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-16T15:04:58.305 INFO:teuthology.orchestra.run.smithi191.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-16T15:04:58.306 INFO:teuthology.orchestra.run.smithi191.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-16T15:04:58.312 INFO:teuthology.orchestra.run.smithi191.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-16T15:04:58.312 INFO:teuthology.orchestra.run.smithi191.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-16T15:04:58.329 INFO:teuthology.orchestra.run.smithi191.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-16T15:04:58.330 INFO:teuthology.orchestra.run.smithi191.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-16T15:04:58.331 INFO:teuthology.orchestra.run.smithi191.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-16T15:04:58.331 INFO:teuthology.orchestra.run.smithi191.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-16T15:04:58.334 INFO:teuthology.orchestra.run.smithi191.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-16T15:04:58.334 INFO:teuthology.orchestra.run.smithi191.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-16T15:04:58.334 INFO:teuthology.orchestra.run.smithi191.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-16T15:04:58.335 INFO:teuthology.orchestra.run.smithi191.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-16T15:04:58.347 INFO:teuthology.orchestra.run.smithi191.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T15:04:58.347 INFO:teuthology.orchestra.run.smithi191.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-16T15:04:58.357 INFO:teuthology.orchestra.run.smithi133.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-16T15:04:58.362 INFO:teuthology.orchestra.run.smithi191.stdout:Get:29 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-845-ge4e227b3-1jammy [3,108 kB] 2024-09-16T15:04:58.364 INFO:teuthology.orchestra.run.smithi191.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-16T15:04:58.366 INFO:teuthology.orchestra.run.smithi191.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-16T15:04:58.367 INFO:teuthology.orchestra.run.smithi191.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-16T15:04:58.367 INFO:teuthology.orchestra.run.smithi191.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-16T15:04:58.373 INFO:teuthology.orchestra.run.smithi191.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-16T15:04:58.373 INFO:teuthology.orchestra.run.smithi191.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-16T15:04:58.374 INFO:teuthology.orchestra.run.smithi191.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-16T15:04:58.375 INFO:teuthology.orchestra.run.smithi191.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-16T15:04:58.382 INFO:teuthology.orchestra.run.smithi191.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-16T15:04:58.384 INFO:teuthology.orchestra.run.smithi191.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-16T15:04:58.394 INFO:teuthology.orchestra.run.smithi133.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-16T15:04:58.394 INFO:teuthology.orchestra.run.smithi191.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-16T15:04:58.396 INFO:teuthology.orchestra.run.smithi191.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-16T15:04:58.396 INFO:teuthology.orchestra.run.smithi191.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-16T15:04:58.397 INFO:teuthology.orchestra.run.smithi133.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-16T15:04:58.399 INFO:teuthology.orchestra.run.smithi191.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-16T15:04:58.399 INFO:teuthology.orchestra.run.smithi191.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-16T15:04:58.400 INFO:teuthology.orchestra.run.smithi191.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-16T15:04:58.403 INFO:teuthology.orchestra.run.smithi191.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-16T15:04:58.406 INFO:teuthology.orchestra.run.smithi191.stdout:Get:47 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-845-ge4e227b3-1jammy [683 kB] 2024-09-16T15:04:58.410 INFO:teuthology.orchestra.run.smithi133.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-845-ge4e227b3-1jammy [3,451 kB] 2024-09-16T15:04:58.412 INFO:teuthology.orchestra.run.smithi133.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-16T15:04:58.416 INFO:teuthology.orchestra.run.smithi191.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-845-ge4e227b3-1jammy [357 kB] 2024-09-16T15:04:58.421 INFO:teuthology.orchestra.run.smithi191.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-845-ge4e227b3-1jammy [32.8 kB] 2024-09-16T15:04:58.422 INFO:teuthology.orchestra.run.smithi191.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-845-ge4e227b3-1jammy [180 kB] 2024-09-16T15:04:58.425 INFO:teuthology.orchestra.run.smithi191.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-845-ge4e227b3-1jammy [63.7 kB] 2024-09-16T15:04:58.426 INFO:teuthology.orchestra.run.smithi191.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-845-ge4e227b3-1jammy [333 kB] 2024-09-16T15:04:58.431 INFO:teuthology.orchestra.run.smithi191.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-845-ge4e227b3-1jammy [5,934 kB] 2024-09-16T15:04:58.457 INFO:teuthology.orchestra.run.smithi191.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-16T15:04:58.458 INFO:teuthology.orchestra.run.smithi191.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-16T15:04:58.459 INFO:teuthology.orchestra.run.smithi191.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-16T15:04:58.459 INFO:teuthology.orchestra.run.smithi191.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-16T15:04:58.460 INFO:teuthology.orchestra.run.smithi191.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-16T15:04:58.460 INFO:teuthology.orchestra.run.smithi191.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-16T15:04:58.461 INFO:teuthology.orchestra.run.smithi191.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-16T15:04:58.463 INFO:teuthology.orchestra.run.smithi191.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T15:04:58.463 INFO:teuthology.orchestra.run.smithi191.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-16T15:04:58.464 INFO:teuthology.orchestra.run.smithi133.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-16T15:04:58.469 INFO:teuthology.orchestra.run.smithi133.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-16T15:04:58.476 INFO:teuthology.orchestra.run.smithi191.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-16T15:04:58.480 INFO:teuthology.orchestra.run.smithi133.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-16T15:04:58.484 INFO:teuthology.orchestra.run.smithi133.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-16T15:04:58.485 INFO:teuthology.orchestra.run.smithi133.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-16T15:04:58.486 INFO:teuthology.orchestra.run.smithi133.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-16T15:04:58.493 INFO:teuthology.orchestra.run.smithi191.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-16T15:04:58.494 INFO:teuthology.orchestra.run.smithi133.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-16T15:04:58.494 INFO:teuthology.orchestra.run.smithi133.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-16T15:04:58.495 INFO:teuthology.orchestra.run.smithi133.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-16T15:04:58.495 INFO:teuthology.orchestra.run.smithi191.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-16T15:04:58.495 INFO:teuthology.orchestra.run.smithi191.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-16T15:04:58.496 INFO:teuthology.orchestra.run.smithi133.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-16T15:04:58.505 INFO:teuthology.orchestra.run.smithi133.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-16T15:04:58.505 INFO:teuthology.orchestra.run.smithi133.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-16T15:04:58.505 INFO:teuthology.orchestra.run.smithi133.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-16T15:04:58.509 INFO:teuthology.orchestra.run.smithi191.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-16T15:04:58.510 INFO:teuthology.orchestra.run.smithi191.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-16T15:04:58.510 INFO:teuthology.orchestra.run.smithi191.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-16T15:04:58.510 INFO:teuthology.orchestra.run.smithi191.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-16T15:04:58.511 INFO:teuthology.orchestra.run.smithi191.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-16T15:04:58.511 INFO:teuthology.orchestra.run.smithi191.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-16T15:04:58.511 INFO:teuthology.orchestra.run.smithi191.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-16T15:04:58.520 INFO:teuthology.orchestra.run.smithi133.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-16T15:04:58.520 INFO:teuthology.orchestra.run.smithi133.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-16T15:04:58.521 INFO:teuthology.orchestra.run.smithi133.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-16T15:04:58.524 INFO:teuthology.orchestra.run.smithi133.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-16T15:04:58.524 INFO:teuthology.orchestra.run.smithi133.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-16T15:04:58.524 INFO:teuthology.orchestra.run.smithi133.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-16T15:04:58.525 INFO:teuthology.orchestra.run.smithi133.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-16T15:04:58.529 INFO:teuthology.orchestra.run.smithi191.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-16T15:04:58.532 INFO:teuthology.orchestra.run.smithi191.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-16T15:04:58.537 INFO:teuthology.orchestra.run.smithi133.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-16T15:04:58.538 INFO:teuthology.orchestra.run.smithi133.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-16T15:04:58.538 INFO:teuthology.orchestra.run.smithi133.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-16T15:04:58.547 INFO:teuthology.orchestra.run.smithi191.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-16T15:04:58.547 INFO:teuthology.orchestra.run.smithi191.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-845-ge4e227b3-1jammy [112 kB] 2024-09-16T15:04:58.548 INFO:teuthology.orchestra.run.smithi191.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-16T15:04:58.549 INFO:teuthology.orchestra.run.smithi191.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-16T15:04:58.549 INFO:teuthology.orchestra.run.smithi191.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-845-ge4e227b3-1jammy [445 kB] 2024-09-16T15:04:58.551 INFO:teuthology.orchestra.run.smithi191.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-16T15:04:58.554 INFO:teuthology.orchestra.run.smithi191.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-16T15:04:58.555 INFO:teuthology.orchestra.run.smithi191.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-845-ge4e227b3-1jammy [22.0 MB] 2024-09-16T15:04:58.555 INFO:teuthology.orchestra.run.smithi133.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-16T15:04:58.556 INFO:teuthology.orchestra.run.smithi133.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-16T15:04:58.556 INFO:teuthology.orchestra.run.smithi133.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-16T15:04:58.558 INFO:teuthology.orchestra.run.smithi191.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-16T15:04:58.561 INFO:teuthology.orchestra.run.smithi191.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T15:04:58.561 INFO:teuthology.orchestra.run.smithi191.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-16T15:04:58.562 INFO:teuthology.orchestra.run.smithi133.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-16T15:04:58.562 INFO:teuthology.orchestra.run.smithi133.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-16T15:04:58.562 INFO:teuthology.orchestra.run.smithi133.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-16T15:04:58.563 INFO:teuthology.orchestra.run.smithi133.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-16T15:04:58.564 INFO:teuthology.orchestra.run.smithi191.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-16T15:04:58.577 INFO:teuthology.orchestra.run.smithi133.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-16T15:04:58.579 INFO:teuthology.orchestra.run.smithi133.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-16T15:04:58.579 INFO:teuthology.orchestra.run.smithi133.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-16T15:04:58.582 INFO:teuthology.orchestra.run.smithi191.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-16T15:04:58.582 INFO:teuthology.orchestra.run.smithi191.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-16T15:04:58.583 INFO:teuthology.orchestra.run.smithi191.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-16T15:04:58.594 INFO:teuthology.orchestra.run.smithi133.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-16T15:04:58.595 INFO:teuthology.orchestra.run.smithi133.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-16T15:04:58.597 INFO:teuthology.orchestra.run.smithi133.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-16T15:04:58.597 INFO:teuthology.orchestra.run.smithi133.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-16T15:04:58.598 INFO:teuthology.orchestra.run.smithi133.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-16T15:04:58.601 INFO:teuthology.orchestra.run.smithi133.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-16T15:04:58.653 INFO:teuthology.orchestra.run.smithi133.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-16T15:04:58.654 INFO:teuthology.orchestra.run.smithi133.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-16T15:04:58.655 INFO:teuthology.orchestra.run.smithi133.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-16T15:04:58.655 INFO:teuthology.orchestra.run.smithi133.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-16T15:04:58.656 INFO:teuthology.orchestra.run.smithi133.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-16T15:04:58.656 INFO:teuthology.orchestra.run.smithi133.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-16T15:04:58.657 INFO:teuthology.orchestra.run.smithi133.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-16T15:04:58.659 INFO:teuthology.orchestra.run.smithi133.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-16T15:04:58.659 INFO:teuthology.orchestra.run.smithi133.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-16T15:04:58.672 INFO:teuthology.orchestra.run.smithi133.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-16T15:04:58.689 INFO:teuthology.orchestra.run.smithi133.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-16T15:04:58.691 INFO:teuthology.orchestra.run.smithi133.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-16T15:04:58.691 INFO:teuthology.orchestra.run.smithi133.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-16T15:04:58.706 INFO:teuthology.orchestra.run.smithi133.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-16T15:04:58.706 INFO:teuthology.orchestra.run.smithi133.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-16T15:04:58.706 INFO:teuthology.orchestra.run.smithi133.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-16T15:04:58.707 INFO:teuthology.orchestra.run.smithi133.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-16T15:04:58.707 INFO:teuthology.orchestra.run.smithi133.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-16T15:04:58.708 INFO:teuthology.orchestra.run.smithi133.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-16T15:04:58.708 INFO:teuthology.orchestra.run.smithi133.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-16T15:04:58.725 INFO:teuthology.orchestra.run.smithi133.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-16T15:04:58.728 INFO:teuthology.orchestra.run.smithi133.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-16T15:04:58.730 INFO:teuthology.orchestra.run.smithi133.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-16T15:04:58.732 INFO:teuthology.orchestra.run.smithi133.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-16T15:04:58.732 INFO:teuthology.orchestra.run.smithi133.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-16T15:04:58.735 INFO:teuthology.orchestra.run.smithi133.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-16T15:04:58.743 INFO:teuthology.orchestra.run.smithi133.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-16T15:04:58.747 INFO:teuthology.orchestra.run.smithi133.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-16T15:04:58.750 INFO:teuthology.orchestra.run.smithi133.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-16T15:04:58.750 INFO:teuthology.orchestra.run.smithi133.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-16T15:04:58.760 INFO:teuthology.orchestra.run.smithi133.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-16T15:04:58.767 INFO:teuthology.orchestra.run.smithi133.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-16T15:04:58.767 INFO:teuthology.orchestra.run.smithi133.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-16T15:04:58.767 INFO:teuthology.orchestra.run.smithi133.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-16T15:04:58.996 INFO:teuthology.orchestra.run.smithi191.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-845-ge4e227b3-1jammy [5,005 kB] 2024-09-16T15:04:59.051 INFO:teuthology.orchestra.run.smithi191.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-845-ge4e227b3-1jammy [241 kB] 2024-09-16T15:04:59.053 INFO:teuthology.orchestra.run.smithi191.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-845-ge4e227b3-1jammy [130 kB] 2024-09-16T15:04:59.055 INFO:teuthology.orchestra.run.smithi191.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-845-ge4e227b3-1jammy [1,494 kB] 2024-09-16T15:04:59.067 INFO:teuthology.orchestra.run.smithi191.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-845-ge4e227b3-1jammy [6,085 kB] 2024-09-16T15:04:59.174 INFO:teuthology.orchestra.run.smithi191.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-845-ge4e227b3-1jammy [22.4 MB] 2024-09-16T15:04:59.531 INFO:teuthology.orchestra.run.smithi133.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-845-ge4e227b3-1jammy [3,108 kB] 2024-09-16T15:04:59.641 INFO:teuthology.orchestra.run.smithi191.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-845-ge4e227b3-1jammy [14.1 kB] 2024-09-16T15:04:59.641 INFO:teuthology.orchestra.run.smithi191.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-845-ge4e227b3-1jammy [834 kB] 2024-09-16T15:04:59.652 INFO:teuthology.orchestra.run.smithi191.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-845-ge4e227b3-1jammy [2,159 kB] 2024-09-16T15:04:59.678 INFO:teuthology.orchestra.run.smithi191.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-845-ge4e227b3-1jammy [240 kB] 2024-09-16T15:04:59.681 INFO:teuthology.orchestra.run.smithi191.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-845-ge4e227b3-1jammy [145 kB] 2024-09-16T15:04:59.683 INFO:teuthology.orchestra.run.smithi191.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-845-ge4e227b3-1jammy [2,233 kB] 2024-09-16T15:04:59.705 INFO:teuthology.orchestra.run.smithi191.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-845-ge4e227b3-1jammy [8,625 kB] 2024-09-16T15:04:59.831 INFO:teuthology.orchestra.run.smithi133.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-845-ge4e227b3-1jammy [683 kB] 2024-09-16T15:04:59.867 INFO:teuthology.orchestra.run.smithi133.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-845-ge4e227b3-1jammy [357 kB] 2024-09-16T15:04:59.872 INFO:teuthology.orchestra.run.smithi191.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-845-ge4e227b3-1jammy [14.3 kB] 2024-09-16T15:04:59.872 INFO:teuthology.orchestra.run.smithi191.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-845-ge4e227b3-1jammy [42.8 MB] 2024-09-16T15:04:59.896 INFO:teuthology.orchestra.run.smithi133.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-845-ge4e227b3-1jammy [32.8 kB] 2024-09-16T15:04:59.897 INFO:teuthology.orchestra.run.smithi133.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-845-ge4e227b3-1jammy [180 kB] 2024-09-16T15:04:59.923 INFO:teuthology.orchestra.run.smithi133.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-845-ge4e227b3-1jammy [63.7 kB] 2024-09-16T15:04:59.925 INFO:teuthology.orchestra.run.smithi133.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-845-ge4e227b3-1jammy [333 kB] 2024-09-16T15:04:59.932 INFO:teuthology.orchestra.run.smithi133.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-845-ge4e227b3-1jammy [5,934 kB] 2024-09-16T15:05:00.286 INFO:teuthology.orchestra.run.smithi133.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-845-ge4e227b3-1jammy [112 kB] 2024-09-16T15:05:00.288 INFO:teuthology.orchestra.run.smithi133.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-845-ge4e227b3-1jammy [445 kB] 2024-09-16T15:05:00.319 INFO:teuthology.orchestra.run.smithi133.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-845-ge4e227b3-1jammy [22.0 MB] 2024-09-16T15:05:01.108 INFO:teuthology.orchestra.run.smithi191.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-845-ge4e227b3-1jammy [122 kB] 2024-09-16T15:05:01.203 INFO:teuthology.orchestra.run.smithi191.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-845-ge4e227b3-1jammy [38.3 kB] 2024-09-16T15:05:01.204 INFO:teuthology.orchestra.run.smithi191.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-845-ge4e227b3-1jammy [11.7 MB] 2024-09-16T15:05:01.206 INFO:teuthology.orchestra.run.smithi133.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-845-ge4e227b3-1jammy [5,005 kB] 2024-09-16T15:05:01.273 INFO:teuthology.orchestra.run.smithi133.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-845-ge4e227b3-1jammy [241 kB] 2024-09-16T15:05:01.276 INFO:teuthology.orchestra.run.smithi133.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-845-ge4e227b3-1jammy [130 kB] 2024-09-16T15:05:01.277 INFO:teuthology.orchestra.run.smithi133.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-845-ge4e227b3-1jammy [1,494 kB] 2024-09-16T15:05:01.306 INFO:teuthology.orchestra.run.smithi133.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-845-ge4e227b3-1jammy [6,085 kB] 2024-09-16T15:05:01.496 INFO:teuthology.orchestra.run.smithi133.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-845-ge4e227b3-1jammy [22.4 MB] 2024-09-16T15:05:01.564 INFO:teuthology.orchestra.run.smithi191.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-845-ge4e227b3-1jammy [93.1 kB] 2024-09-16T15:05:01.937 INFO:teuthology.orchestra.run.smithi133.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-845-ge4e227b3-1jammy [14.1 kB] 2024-09-16T15:05:01.937 INFO:teuthology.orchestra.run.smithi133.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-845-ge4e227b3-1jammy [834 kB] 2024-09-16T15:05:01.945 INFO:teuthology.orchestra.run.smithi133.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-845-ge4e227b3-1jammy [2,159 kB] 2024-09-16T15:05:01.969 INFO:teuthology.orchestra.run.smithi133.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-845-ge4e227b3-1jammy [240 kB] 2024-09-16T15:05:01.972 INFO:teuthology.orchestra.run.smithi133.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-845-ge4e227b3-1jammy [145 kB] 2024-09-16T15:05:01.976 INFO:teuthology.orchestra.run.smithi133.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-845-ge4e227b3-1jammy [2,233 kB] 2024-09-16T15:05:01.995 INFO:teuthology.orchestra.run.smithi133.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-845-ge4e227b3-1jammy [8,625 kB] 2024-09-16T15:05:02.141 INFO:teuthology.orchestra.run.smithi191.stdout:Fetched 163 MB in 4s (46.3 MB/s) 2024-09-16T15:05:02.163 INFO:teuthology.orchestra.run.smithi133.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-845-ge4e227b3-1jammy [14.3 kB] 2024-09-16T15:05:02.164 INFO:teuthology.orchestra.run.smithi133.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-845-ge4e227b3-1jammy [42.8 MB] 2024-09-16T15:05:02.312 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-16T15:05:02.405 INFO:teuthology.orchestra.run.smithi191.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-16T15:05:02.410 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-16T15:05:02.511 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T15:05:02.882 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-16T15:05:02.908 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-16T15:05:02.948 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T15:05:02.999 INFO:teuthology.orchestra.run.smithi133.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-845-ge4e227b3-1jammy [122 kB] 2024-09-16T15:05:03.004 INFO:teuthology.orchestra.run.smithi133.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-845-ge4e227b3-1jammy [38.3 kB] 2024-09-16T15:05:03.005 INFO:teuthology.orchestra.run.smithi133.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-845-ge4e227b3-1jammy [11.7 MB] 2024-09-16T15:05:03.151 INFO:teuthology.orchestra.run.smithi133.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-845-ge4e227b3-1jammy [93.1 kB] 2024-09-16T15:05:03.345 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T15:05:03.371 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-16T15:05:03.420 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T15:05:03.677 INFO:teuthology.orchestra.run.smithi133.stdout:Fetched 163 MB in 5s (33.2 MB/s) 2024-09-16T15:05:03.832 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T15:05:03.842 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-16T15:05:03.858 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-16T15:05:03.906 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T15:05:03.942 INFO:teuthology.orchestra.run.smithi133.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-16T15:05:03.948 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-16T15:05:04.024 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T15:05:04.285 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-16T15:05:04.311 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-16T15:05:04.351 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T15:05:04.447 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T15:05:04.473 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T15:05:04.528 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:05:04.697 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-16T15:05:04.723 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-16T15:05:04.754 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T15:05:05.016 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T15:05:05.042 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T15:05:05.091 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:05:05.110 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-16T15:05:05.136 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-16T15:05:05.307 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T15:05:05.478 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T15:05:05.505 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T15:05:05.553 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:05:05.753 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-16T15:05:05.779 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T15:05:05.827 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:05:05.866 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-16T15:05:05.892 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-16T15:05:05.932 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T15:05:06.189 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-16T15:05:06.216 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T15:05:06.247 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:05:06.278 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librados2. 2024-09-16T15:05:06.304 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../008-librados2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:06.344 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:06.575 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-16T15:05:06.602 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-16T15:05:06.641 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:05:06.865 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librbd1. 2024-09-16T15:05:06.891 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../009-librbd1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:06.940 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:06.995 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-16T15:05:07.021 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-16T15:05:07.061 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T15:05:07.356 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librados2. 2024-09-16T15:05:07.371 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../008-librados2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:07.394 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T15:05:07.405 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:07.420 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:07.460 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:07.713 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rados. 2024-09-16T15:05:07.729 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../011-python3-rados_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:07.772 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:07.833 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librbd1. 2024-09-16T15:05:07.860 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../009-librbd1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:07.925 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:08.084 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T15:05:08.111 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:08.150 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:08.353 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libcephfs2. 2024-09-16T15:05:08.379 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:08.403 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T15:05:08.419 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:08.419 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:08.462 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:08.705 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rados. 2024-09-16T15:05:08.720 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../011-python3-rados_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:08.757 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T15:05:08.763 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:08.784 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:08.824 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:09.057 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-16T15:05:09.084 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:09.124 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:09.161 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T15:05:09.188 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-16T15:05:09.228 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T15:05:09.326 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cephfs. 2024-09-16T15:05:09.339 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:09.376 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:09.540 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T15:05:09.567 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-16T15:05:09.607 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-16T15:05:09.688 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-16T15:05:09.714 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:09.754 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:09.843 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T15:05:09.859 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:09.893 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:10.074 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-16T15:05:10.100 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-16T15:05:10.140 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T15:05:10.231 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T15:05:10.258 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-16T15:05:10.297 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T15:05:10.460 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-prettytable. 2024-09-16T15:05:10.486 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-16T15:05:10.526 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-16T15:05:10.618 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package librgw2. 2024-09-16T15:05:10.644 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../019-librgw2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:10.684 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:10.753 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rbd. 2024-09-16T15:05:10.769 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:10.803 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:11.146 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T15:05:11.148 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-16T15:05:11.161 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:11.174 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-16T15:05:11.204 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:11.214 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T15:05:11.526 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package librgw2. 2024-09-16T15:05:11.541 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-16T15:05:11.552 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../019-librgw2_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:11.568 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-16T15:05:11.592 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:11.719 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T15:05:12.103 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T15:05:12.119 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rgw. 2024-09-16T15:05:12.130 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:12.146 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:12.161 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:12.186 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:12.472 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-16T15:05:12.498 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-16T15:05:12.530 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T15:05:12.557 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-common. 2024-09-16T15:05:12.583 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../023-ceph-common_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:12.641 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:12.833 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libradosstriper1. 2024-09-16T15:05:12.860 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:12.899 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:13.336 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-common. 2024-09-16T15:05:13.363 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../023-ceph-common_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:13.403 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:13.932 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-base. 2024-09-16T15:05:13.957 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../024-ceph-base_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:14.007 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:14.612 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-16T15:05:14.640 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-16T15:05:14.679 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-16T15:05:14.683 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-base. 2024-09-16T15:05:14.710 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../024-ceph-base_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:14.759 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:15.008 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cheroot. 2024-09-16T15:05:15.034 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-16T15:05:15.066 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T15:05:15.361 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-16T15:05:15.388 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-16T15:05:15.428 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-16T15:05:15.521 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-16T15:05:15.547 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-16T15:05:15.579 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-16T15:05:15.732 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-16T15:05:15.758 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-16T15:05:15.798 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-16T15:05:15.882 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cheroot. 2024-09-16T15:05:15.908 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-16T15:05:15.948 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T15:05:16.111 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-16T15:05:16.137 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-16T15:05:16.177 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-16T15:05:16.293 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-16T15:05:16.314 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-16T15:05:16.351 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-16T15:05:16.456 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-tempora. 2024-09-16T15:05:16.483 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-16T15:05:16.522 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-16T15:05:16.629 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-16T15:05:16.655 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-16T15:05:16.687 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-16T15:05:16.851 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-portend. 2024-09-16T15:05:16.878 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-16T15:05:16.918 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-16T15:05:16.990 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-16T15:05:17.016 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-16T15:05:17.056 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-16T15:05:17.342 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-tempora. 2024-09-16T15:05:17.369 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-16T15:05:17.389 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-16T15:05:17.409 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-16T15:05:17.415 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-16T15:05:17.447 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-16T15:05:17.695 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-portend. 2024-09-16T15:05:17.721 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-16T15:05:17.750 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T15:05:17.762 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-16T15:05:17.777 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-16T15:05:17.825 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-16T15:05:18.031 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-16T15:05:18.057 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-16T15:05:18.097 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-16T15:05:18.172 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T15:05:18.198 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-16T15:05:18.238 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-16T15:05:18.400 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-16T15:05:18.427 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-16T15:05:18.467 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-16T15:05:18.526 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T15:05:18.552 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-16T15:05:18.592 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-16T15:05:18.779 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-natsort. 2024-09-16T15:05:18.801 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-16T15:05:18.837 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-16T15:05:18.896 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T15:05:18.923 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-16T15:05:18.963 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-16T15:05:19.173 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-logutils. 2024-09-16T15:05:19.199 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-16T15:05:19.225 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-mako. 2024-09-16T15:05:19.240 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-16T15:05:19.241 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-16T15:05:19.283 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T15:05:19.543 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-16T15:05:19.569 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-16T15:05:19.587 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T15:05:19.609 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-16T15:05:19.614 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T15:05:19.654 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T15:05:19.920 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-mako. 2024-09-16T15:05:19.946 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-16T15:05:19.957 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T15:05:19.984 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-16T15:05:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T15:05:20.024 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-16T15:05:20.278 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-webob. 2024-09-16T15:05:20.293 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-16T15:05:20.307 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-16T15:05:20.333 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-16T15:05:20.336 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T15:05:20.373 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-16T15:05:20.643 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-16T15:05:20.648 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T15:05:20.669 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-16T15:05:20.675 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-16T15:05:20.709 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-16T15:05:20.718 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T15:05:21.012 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-webob. 2024-09-16T15:05:21.019 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T15:05:21.038 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-16T15:05:21.046 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-16T15:05:21.077 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T15:05:21.078 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T15:05:21.364 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-paste. 2024-09-16T15:05:21.391 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-16T15:05:21.423 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-waitress. 2024-09-16T15:05:21.431 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T15:05:21.449 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-16T15:05:21.493 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T15:05:21.727 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T15:05:21.753 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-16T15:05:21.793 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T15:05:21.801 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-tempita. 2024-09-16T15:05:21.827 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-16T15:05:21.867 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T15:05:22.055 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T15:05:22.082 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-16T15:05:22.113 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-16T15:05:22.137 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-paste. 2024-09-16T15:05:22.163 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-16T15:05:22.195 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T15:05:22.481 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-16T15:05:22.497 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-16T15:05:22.531 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T15:05:22.559 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T15:05:22.586 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-16T15:05:22.626 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-16T15:05:22.826 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-16T15:05:22.852 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-16T15:05:22.888 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T15:05:22.892 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-16T15:05:22.914 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-16T15:05:22.946 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T15:05:23.195 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-webtest. 2024-09-16T15:05:23.223 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-16T15:05:23.261 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-16T15:05:23.269 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-certifi. 2024-09-16T15:05:23.296 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-16T15:05:23.327 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-16T15:05:23.573 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pecan. 2024-09-16T15:05:23.599 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-16T15:05:23.639 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T15:05:23.639 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-idna. 2024-09-16T15:05:23.666 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-16T15:05:23.706 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T15:05:24.002 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-certifi. 2024-09-16T15:05:24.010 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-urllib3. 2024-09-16T15:05:24.028 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-16T15:05:24.036 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-16T15:05:24.088 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-16T15:05:24.088 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T15:05:24.389 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-requests. 2024-09-16T15:05:24.397 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-idna. 2024-09-16T15:05:24.415 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-16T15:05:24.423 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-16T15:05:24.455 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T15:05:24.455 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T15:05:24.726 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T15:05:24.749 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-16T15:05:24.774 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-urllib3. 2024-09-16T15:05:24.784 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T15:05:24.801 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-16T15:05:24.840 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T15:05:25.105 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T15:05:25.120 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:25.155 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:25.160 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-requests. 2024-09-16T15:05:25.187 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-16T15:05:25.227 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T15:05:25.545 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T15:05:25.572 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:25.584 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-16T15:05:25.611 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-16T15:05:25.611 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T15:05:25.924 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T15:05:25.950 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-16T15:05:25.990 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-16T15:05:26.007 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-16T15:05:26.022 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:26.066 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:26.260 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T15:05:26.276 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-16T15:05:26.310 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-16T15:05:26.456 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-16T15:05:26.482 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:26.522 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:26.639 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T15:05:26.666 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:26.706 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:26.833 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-16T15:05:26.860 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-16T15:05:26.917 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-16T15:05:27.077 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T15:05:27.103 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:27.143 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:27.219 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-openssl. 2024-09-16T15:05:27.246 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-16T15:05:27.286 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-16T15:05:27.623 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr. 2024-09-16T15:05:27.649 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:27.689 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:27.776 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T15:05:27.803 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:27.898 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:28.084 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mon. 2024-09-16T15:05:28.110 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:28.150 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:28.704 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-osd. 2024-09-16T15:05:28.730 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:28.770 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:28.944 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph. 2024-09-16T15:05:28.959 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../060-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:29.019 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:29.339 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T15:05:29.366 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:29.406 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:29.735 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T15:05:29.761 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:29.793 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:29.807 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph. 2024-09-16T15:05:29.822 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../060-ceph_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:29.865 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:30.189 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package cephadm. 2024-09-16T15:05:30.193 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-fuse. 2024-09-16T15:05:30.216 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../063-cephadm_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:30.219 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:30.255 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:30.259 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:30.568 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-16T15:05:30.587 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mds. 2024-09-16T15:05:30.594 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-16T15:05:30.614 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:30.634 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-16T15:05:30.646 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:30.972 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-16T15:05:30.998 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T15:05:31.024 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package cephadm. 2024-09-16T15:05:31.038 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T15:05:31.050 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../063-cephadm_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:31.090 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:31.402 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-16T15:05:31.428 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-16T15:05:31.851 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-16T15:05:32.513 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-16T15:05:32.528 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T15:05:32.571 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T15:05:32.806 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-babel. 2024-09-16T15:05:32.833 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T15:05:32.871 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T15:05:33.216 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T15:05:33.243 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-16T15:05:33.283 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T15:05:33.526 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-babel. 2024-09-16T15:05:33.553 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-16T15:05:33.592 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T15:05:33.636 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T15:05:33.662 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:33.702 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:33.912 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-jinja2. 2024-09-16T15:05:33.939 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-16T15:05:33.979 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T15:05:34.056 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T15:05:34.082 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-16T15:05:34.122 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-16T15:05:34.324 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-16T15:05:34.350 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:34.390 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:34.408 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-py. 2024-09-16T15:05:34.435 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-16T15:05:34.475 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-16T15:05:34.727 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pluggy. 2024-09-16T15:05:34.754 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-16T15:05:34.770 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-16T15:05:34.794 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-16T15:05:34.797 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-16T15:05:34.828 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-16T15:05:35.105 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-py. 2024-09-16T15:05:35.115 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-toml. 2024-09-16T15:05:35.131 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-16T15:05:35.141 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-16T15:05:35.180 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-16T15:05:35.181 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-16T15:05:35.484 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T15:05:35.500 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-16T15:05:35.511 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-16T15:05:35.527 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-16T15:05:35.550 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T15:05:35.566 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-16T15:05:35.862 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T15:05:35.889 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T15:05:35.920 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T15:05:36.095 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-toml. 2024-09-16T15:05:36.121 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-16T15:05:36.153 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-16T15:05:36.215 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-routes. 2024-09-16T15:05:36.241 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-16T15:05:36.281 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T15:05:36.464 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pytest. 2024-09-16T15:05:36.491 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-16T15:05:36.530 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T15:05:36.585 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T15:05:36.612 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:36.651 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:36.850 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-16T15:05:36.866 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-16T15:05:36.900 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-16T15:05:37.203 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-routes. 2024-09-16T15:05:37.230 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-16T15:05:37.270 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T15:05:37.624 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-16T15:05:37.650 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:37.690 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:38.240 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-16T15:05:38.267 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-16T15:05:38.306 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T15:05:38.785 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T15:05:38.811 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-16T15:05:38.851 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T15:05:38.886 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-16T15:05:38.902 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-16T15:05:38.945 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T15:05:39.213 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-16T15:05:39.240 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-16T15:05:39.279 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-16T15:05:39.382 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-joblib. 2024-09-16T15:05:39.397 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-16T15:05:39.431 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T15:05:39.582 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T15:05:39.609 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-16T15:05:39.648 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T15:05:39.802 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-16T15:05:39.817 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-16T15:05:39.860 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-16T15:05:40.163 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-sklearn. 2024-09-16T15:05:40.190 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-16T15:05:40.229 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T15:05:40.365 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T15:05:40.393 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:40.431 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:41.005 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-16T15:05:41.032 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:41.071 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:41.434 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T15:05:41.462 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-16T15:05:41.501 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-16T15:05:41.905 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T15:05:41.932 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-16T15:05:41.963 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-16T15:05:41.966 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-cachetools. 2024-09-16T15:05:41.992 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-16T15:05:42.032 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-16T15:05:42.317 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T15:05:42.344 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-16T15:05:42.383 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T15:05:42.411 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-16T15:05:42.438 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-16T15:05:42.478 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-16T15:05:42.695 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T15:05:42.721 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-16T15:05:42.761 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-16T15:05:42.823 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-16T15:05:42.850 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-16T15:05:42.890 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T15:05:43.057 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T15:05:43.084 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-16T15:05:43.115 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-16T15:05:43.176 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-rsa. 2024-09-16T15:05:43.203 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-16T15:05:43.242 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-16T15:05:43.376 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T15:05:43.392 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-16T15:05:43.434 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T15:05:43.563 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-google-auth. 2024-09-16T15:05:43.590 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-16T15:05:43.629 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-16T15:05:43.737 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T15:05:43.764 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-16T15:05:43.803 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-16T15:05:43.933 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-16T15:05:43.959 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-16T15:05:43.999 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T15:05:44.107 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T15:05:44.135 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-16T15:05:44.173 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T15:05:44.302 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-websocket. 2024-09-16T15:05:44.329 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-16T15:05:44.368 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-16T15:05:44.680 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-16T15:05:44.707 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-16T15:05:44.771 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T15:05:44.795 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T15:05:44.811 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:44.845 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:45.156 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T15:05:45.183 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-16T15:05:45.223 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T15:05:45.469 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-16T15:05:45.496 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:45.535 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:45.592 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T15:05:45.620 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T15:05:45.659 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T15:05:45.879 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-16T15:05:45.906 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-16T15:05:45.911 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package jq. 2024-09-16T15:05:45.927 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T15:05:45.946 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T15:05:45.961 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-16T15:05:46.180 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package socat. 2024-09-16T15:05:46.196 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-16T15:05:46.230 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-16T15:05:46.429 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-16T15:05:46.457 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T15:05:46.517 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T15:05:46.540 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T15:05:46.544 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-16T15:05:46.583 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-16T15:05:46.768 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package jq. 2024-09-16T15:05:46.784 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-16T15:05:46.818 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-16T15:05:46.929 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-test. 2024-09-16T15:05:46.949 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../096-ceph-test_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:46.987 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:47.070 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package socat. 2024-09-16T15:05:47.094 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-16T15:05:47.137 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-16T15:05:47.441 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package xmlstarlet. 2024-09-16T15:05:47.467 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-16T15:05:47.506 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-16T15:05:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-test. 2024-09-16T15:05:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../096-ceph-test_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:49.133 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T15:05:49.160 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:49.199 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:49.520 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T15:05:49.547 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:49.577 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:49.823 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package ceph-volume. 2024-09-16T15:05:49.851 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-845-ge4e227b3-1jammy_all.deb ... 2024-09-16T15:05:49.890 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:49.923 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T15:05:49.950 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-16T15:05:49.989 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T15:05:50.169 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-16T15:05:50.185 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:50.218 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:50.344 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-16T15:05:50.371 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-16T15:05:50.410 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-16T15:05:50.580 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package nvme-cli. 2024-09-16T15:05:50.607 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-16T15:05:50.646 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T15:05:50.847 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T15:05:50.874 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T15:05:50.913 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T15:05:50.985 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-16T15:05:51.001 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-16T15:05:51.043 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-16T15:05:51.225 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T15:05:51.252 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-16T15:05:51.283 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-16T15:05:51.454 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pastescript. 2024-09-16T15:05:51.482 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-16T15:05:51.520 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-16T15:05:51.619 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pygments. 2024-09-16T15:05:51.647 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-16T15:05:51.694 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T15:05:51.850 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-psutil. 2024-09-16T15:05:51.877 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-16T15:05:51.916 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-16T15:05:52.123 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T15:05:52.150 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-16T15:05:52.189 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-16T15:05:52.236 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pygments. 2024-09-16T15:05:52.263 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-16T15:05:52.302 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T15:05:52.500 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T15:05:52.528 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-16T15:05:52.566 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-16T15:05:52.756 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-16T15:05:52.784 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-16T15:05:52.822 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-16T15:05:52.852 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T15:05:52.868 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-16T15:05:52.910 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-16T15:05:53.284 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package python3-simplejson. 2024-09-16T15:05:53.299 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-16T15:05:53.333 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-16T15:05:53.578 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-16T15:05:53.594 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-16T15:05:53.636 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-16T15:05:53.690 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package radosgw. 2024-09-16T15:05:53.717 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../107-radosgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:53.756 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:54.348 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package radosgw. 2024-09-16T15:05:54.376 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../107-radosgw_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:54.414 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:54.467 INFO:teuthology.orchestra.run.smithi191.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T15:05:54.483 INFO:teuthology.orchestra.run.smithi191.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:54.517 INFO:teuthology.orchestra.run.smithi191.stdout:Unpacking rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:54.813 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-16T15:05:55.088 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T15:05:55.168 INFO:teuthology.orchestra.run.smithi133.stdout:Selecting previously unselected package rbd-fuse. 2024-09-16T15:05:55.195 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-845-ge4e227b3-1jammy_amd64.deb ... 2024-09-16T15:05:55.214 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T15:05:55.234 INFO:teuthology.orchestra.run.smithi133.stdout:Unpacking rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:55.475 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-16T15:05:55.537 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-16T15:05:55.789 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-16T15:05:55.827 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-16T15:05:55.915 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-16T15:05:56.176 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-16T15:05:56.248 INFO:teuthology.orchestra.run.smithi191.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-16T15:05:56.316 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:56.488 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-16T15:05:56.857 INFO:teuthology.orchestra.run.smithi191.stdout:Adding system user cephadm....done 2024-09-16T15:05:56.901 INFO:teuthology.orchestra.run.smithi133.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-16T15:05:56.972 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T15:05:57.058 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:05:57.240 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-16T15:05:57.483 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-16T15:05:57.607 INFO:teuthology.orchestra.run.smithi133.stdout:Adding system user cephadm....done 2024-09-16T15:05:57.608 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-16T15:05:57.739 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-16T15:05:57.885 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T15:05:58.024 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-16T15:05:58.136 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T15:05:58.237 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-16T15:05:58.275 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-16T15:05:58.401 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-16T15:05:58.530 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T15:05:58.668 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-16T15:05:58.907 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-16T15:05:58.936 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-16T15:05:59.054 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-16T15:05:59.356 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-16T15:05:59.408 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-16T15:05:59.667 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-16T15:05:59.715 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-16T15:05:59.918 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-16T15:05:59.975 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-16T15:06:00.177 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-16T15:06:00.251 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-16T15:06:00.428 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:00.494 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-16T15:06:00.688 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T15:06:00.728 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-16T15:06:00.814 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-16T15:06:00.962 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-ceph-argparse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:01.065 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T15:06:01.191 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-16T15:06:01.230 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-16T15:06:01.356 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-16T15:06:01.567 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T15:06:01.616 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-16T15:06:01.693 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-16T15:06:01.742 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-16T15:06:01.952 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T15:06:02.101 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-16T15:06:02.203 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-16T15:06:02.210 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-16T15:06:02.453 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-16T15:06:02.505 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T15:06:02.696 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-16T15:06:03.014 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-16T15:06:03.198 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T15:06:03.449 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T15:06:03.567 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-16T15:06:03.674 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-16T15:06:03.693 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T15:06:03.934 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-16T15:06:03.985 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-16T15:06:04.060 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-16T15:06:04.178 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-16T15:06:04.370 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-16T15:06:04.462 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-16T15:06:04.613 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T15:06:04.847 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-16T15:06:04.989 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T15:06:05.090 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-16T15:06:05.274 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T15:06:05.474 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-16T15:06:05.617 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-16T15:06:05.784 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-16T15:06:05.882 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-16T15:06:05.994 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:06.118 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-16T15:06:06.278 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T15:06:06.353 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-16T15:06:06.454 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:06.604 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T15:06:06.739 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-16T15:06:07.065 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-16T15:06:07.735 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:06:07.877 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-16T15:06:08.145 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T15:06:08.204 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:06:08.271 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-16T15:06:08.338 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-16T15:06:08.397 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T15:06:08.597 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-16T15:06:08.690 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-16T15:06:08.724 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-16T15:06:08.850 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-16T15:06:08.941 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-16T15:06:09.143 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-16T15:06:09.183 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:06:09.310 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T15:06:09.385 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-16T15:06:09.435 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-16T15:06:09.628 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:06:09.745 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-16T15:06:09.779 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-16T15:06:09.914 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-16T15:06:10.022 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-16T15:06:10.215 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-16T15:06:10.273 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-16T15:06:10.474 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-16T15:06:10.515 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T15:06:10.733 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-16T15:06:10.758 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-16T15:06:10.993 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-16T15:06:11.017 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T15:06:11.143 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-16T15:06:11.227 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-16T15:06:11.419 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T15:06:11.478 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-16T15:06:11.554 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T15:06:11.604 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-16T15:06:11.840 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T15:06:11.883 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-16T15:06:12.008 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-16T15:06:12.139 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-16T15:06:12.293 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-16T15:06:12.382 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T15:06:12.586 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-16T15:06:12.764 INFO:teuthology.orchestra.run.smithi191.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-16T15:06:12.836 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-16T15:06:13.009 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T15:06:13.135 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T15:06:13.316 INFO:teuthology.orchestra.run.smithi133.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-16T15:06:13.520 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-16T15:06:13.564 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-16T15:06:13.690 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-16T15:06:13.780 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-16T15:06:14.031 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:06:14.075 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-16T15:06:14.173 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T15:06:14.343 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-16T15:06:14.474 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T15:06:14.602 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-16T15:06:14.728 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-16T15:06:14.800 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T15:06:15.046 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-16T15:06:15.068 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-16T15:06:15.194 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-16T15:06:15.339 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-16T15:06:15.470 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-16T15:06:15.623 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-16T15:06:15.738 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T15:06:15.749 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-16T15:06:16.006 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-16T15:06:16.016 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-16T15:06:16.276 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-16T15:06:16.399 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T15:06:16.535 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-16T15:06:16.700 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T15:06:16.912 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-16T15:06:17.035 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T15:06:17.136 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:17.196 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-16T15:06:17.262 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:17.388 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:17.514 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:17.540 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-16T15:06:17.640 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:17.666 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librados2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:17.792 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librgw2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:17.826 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-16T15:06:17.918 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:18.036 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libcephfs2 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:18.086 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:18.145 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libradosstriper1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:18.229 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:18.339 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-16T15:06:18.355 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:18.624 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up librbd1 (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:18.642 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T15:06:18.642 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T15:06:18.751 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:18.876 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:19.085 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T15:06:19.085 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-16T15:06:19.115 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:19.224 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:19.350 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T15:06:19.603 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up libcephfs-dev (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:19.729 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rados (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:19.855 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-16T15:06:20.588 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:20.714 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:20.831 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:20.941 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:21.009 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rbd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:21.067 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:21.135 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up rbd-fuse (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:21.244 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-rgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:21.345 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up python3-cephfs (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:21.471 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-common (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:21.601 INFO:teuthology.orchestra.run.smithi191.stdout:Adding group ceph....done 2024-09-16T15:06:21.969 INFO:teuthology.orchestra.run.smithi191.stdout:Adding system user ceph....done 2024-09-16T15:06:22.040 INFO:teuthology.orchestra.run.smithi133.stdout:Adding group ceph....done 2024-09-16T15:06:22.137 INFO:teuthology.orchestra.run.smithi191.stdout:Setting system user ceph properties....done 2024-09-16T15:06:22.157 INFO:teuthology.orchestra.run.smithi191.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T15:06:22.292 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T15:06:22.398 INFO:teuthology.orchestra.run.smithi133.stdout:Adding system user ceph....done 2024-09-16T15:06:22.559 INFO:teuthology.orchestra.run.smithi133.stdout:Setting system user ceph properties....done 2024-09-16T15:06:22.571 INFO:teuthology.orchestra.run.smithi133.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-16T15:06:22.603 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T15:06:22.707 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-16T15:06:23.006 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-16T15:06:23.076 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:23.177 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:23.455 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-test (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:23.607 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up radosgw (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:23.639 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:06:23.639 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:06:24.069 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:06:24.070 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:06:24.139 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:24.491 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T15:06:24.533 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-base (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:24.886 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-16T15:06:24.982 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:25.216 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T15:06:25.216 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T15:06:25.368 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mds (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:25.573 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T15:06:25.573 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-16T15:06:25.709 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:25.935 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T15:06:25.936 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T15:06:26.045 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:26.249 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T15:06:26.249 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-16T15:06:26.435 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:26.713 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T15:06:26.713 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T15:06:26.729 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-osd (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:27.008 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T15:06:27.008 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-16T15:06:27.237 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:27.371 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:27.473 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:27.514 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:27.710 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T15:06:27.710 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T15:06:27.899 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:28.050 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mon (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:28.182 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:28.262 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T15:06:28.262 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-16T15:06:28.333 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:28.459 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:28.601 INFO:teuthology.orchestra.run.smithi191.stdout:Setting up ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:28.759 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:28.886 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:28.929 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-16T15:06:29.003 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:29.137 INFO:teuthology.orchestra.run.smithi133.stdout:Setting up ceph-volume (18.2.4-845-ge4e227b3-1jammy) ... 2024-09-16T15:06:29.490 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-16T15:06:29.922 INFO:teuthology.orchestra.run.smithi191.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-16T15:06:30.374 INFO:teuthology.orchestra.run.smithi191.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T15:06:30.379 DEBUG:teuthology.parallel:result is None 2024-09-16T15:06:30.591 INFO:teuthology.orchestra.run.smithi133.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-16T15:06:31.070 INFO:teuthology.orchestra.run.smithi133.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-16T15:06:31.074 DEBUG:teuthology.parallel:result is None 2024-09-16T15:06:31.075 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T15:06:31.214 DEBUG:teuthology.orchestra.run.smithi133:> dpkg-query -W -f '${Version}' ceph 2024-09-16T15:06:31.231 INFO:teuthology.orchestra.run.smithi133.stdout:18.2.4-845-ge4e227b3-1jammy 2024-09-16T15:06:31.231 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T15:06:31.231 INFO:teuthology.task.install:The correct ceph version 18.2.4-845-ge4e227b3-1jammy is installed. 2024-09-16T15:06:31.233 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T15:06:31.373 DEBUG:teuthology.orchestra.run.smithi191:> dpkg-query -W -f '${Version}' ceph 2024-09-16T15:06:31.392 INFO:teuthology.orchestra.run.smithi191.stdout:18.2.4-845-ge4e227b3-1jammy 2024-09-16T15:06:31.392 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845-ge4e227b3-1jammy 2024-09-16T15:06:31.392 INFO:teuthology.task.install:The correct ceph version 18.2.4-845-ge4e227b3-1jammy is installed. 2024-09-16T15:06:31.394 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T15:06:31.394 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:31.394 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T15:06:31.409 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:31.409 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T15:06:31.449 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T15:06:31.449 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:31.449 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T15:06:31.465 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T15:06:31.521 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:31.521 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T15:06:31.534 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T15:06:31.589 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T15:06:31.589 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:31.589 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T15:06:31.602 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T15:06:31.656 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:31.656 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T15:06:31.669 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T15:06:31.723 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T15:06:31.723 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:31.724 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T15:06:31.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T15:06:31.789 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:31.789 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T15:06:31.801 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T15:06:31.855 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T15:06:31.924 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T15:06:31.924 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /var/log/ceph 2024-09-16T15:06:31.927 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /var/log/ceph 2024-09-16T15:06:31.938 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T15:06:31.939 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T15:06:31.983 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T15:06:31.995 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T15:06:31.995 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T15:06:32.035 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T15:06:32.054 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-16T15:06:32.054 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-16T15:06:32.054 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-16T15:06:32.054 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:32.054 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-16T15:06:32.094 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-16T15:06:32.169 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:32.169 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-16T15:06:32.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-16T15:06:32.253 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T15:06:32.253 INFO:tasks.ceph:config {'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 0}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_', 'replacing it with standby', 'No standby daemons available', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T15:06:32.253 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795', 'branch': 'reef', 'description': 'rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/workunits}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7906795', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 2, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 0}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 931, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'], ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1']], 'seed': 1390, 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'sleep_before_teardown': 0, 'subset': '5319/100000', 'suite': 'rados', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6', 'targets': {'smithi133.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ0hLxykiCI0jl6iX32iCl4ULylUNnJXigJVoSgYcwXwuyBuQh4A/AJ76JEgMXRdCWEYR45us25VmrYCffVPdwE=', 'smithi191.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNckmitjRQ9MZ/bZKvbOthOOBHXpJb9piMQfQSWEhqEA3dvN2/MeIK+QQUyj/snI7jbBkHJb/jmULZ4+sH4RFOc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_', 'replacing it with standby', 'No standby daemons available', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'wait-for-scrub': False, 'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 0}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['mgr/test_localpool.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_22:00:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T15:06:32.253 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T15:06:32.256 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T15:06:32.297 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T15:06:32.303 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T15:06:32.357 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:32.357 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T15:06:32.363 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T15:06:32.363 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-09-16T15:06:32.410 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T15:06:32.410 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:06:32.410 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 877 Links: 1 2024-09-16T15:06:32.410 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:06:32.410 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-16 15:01:14.556252144 +0000 2024-09-16T15:06:32.410 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-16 15:01:14.300254418 +0000 2024-09-16T15:06:32.410 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-16 15:01:14.300254418 +0000 2024-09-16T15:06:32.410 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-09-16T15:06:32.410 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T15:06:32.463 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-16T15:06:32.464 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-16T15:06:32.464 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000345599 s, 1.5 MB/s 2024-09-16T15:06:32.465 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T15:06:32.515 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-09-16T15:06:32.561 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T15:06:32.561 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:06:32.562 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-09-16T15:06:32.562 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:06:32.562 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-16 15:01:15.048247774 +0000 2024-09-16T15:06:32.562 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-16 15:01:14.808249905 +0000 2024-09-16T15:06:32.562 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-16 15:01:14.808249905 +0000 2024-09-16T15:06:32.562 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-09-16T15:06:32.562 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T15:06:32.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-16T15:06:32.616 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-16T15:06:32.616 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.00042104 s, 1.2 MB/s 2024-09-16T15:06:32.617 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T15:06:32.667 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-09-16T15:06:32.714 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T15:06:32.714 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:06:32.714 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 913 Links: 1 2024-09-16T15:06:32.714 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:06:32.714 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-16 15:01:15.512243651 +0000 2024-09-16T15:06:32.714 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-16 15:01:15.276245748 +0000 2024-09-16T15:06:32.714 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-16 15:01:15.276245748 +0000 2024-09-16T15:06:32.714 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-09-16T15:06:32.714 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T15:06:32.767 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-16T15:06:32.767 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-16T15:06:32.768 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000406709 s, 1.3 MB/s 2024-09-16T15:06:32.769 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T15:06:32.819 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-09-16T15:06:32.866 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T15:06:32.866 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:06:32.866 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 936 Links: 1 2024-09-16T15:06:32.866 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:06:32.866 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-16 15:01:15.980239494 +0000 2024-09-16T15:06:32.866 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-16 15:01:15.736241661 +0000 2024-09-16T15:06:32.866 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-16 15:01:15.736241661 +0000 2024-09-16T15:06:32.866 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: - 2024-09-16T15:06:32.866 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T15:06:32.919 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-16T15:06:32.919 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-16T15:06:32.919 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000300311 s, 1.7 MB/s 2024-09-16T15:06:32.921 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T15:06:32.971 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-09-16T15:06:32.971 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:32.971 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T15:06:32.977 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T15:06:32.977 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-09-16T15:06:33.025 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T15:06:33.025 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:06:33.026 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 885 Links: 1 2024-09-16T15:06:33.026 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:06:33.026 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-16 15:01:16.562983626 +0000 2024-09-16T15:06:33.026 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-16 15:01:16.318985805 +0000 2024-09-16T15:06:33.026 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-16 15:01:16.318985805 +0000 2024-09-16T15:06:33.026 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-09-16T15:06:33.026 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T15:06:33.080 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-16T15:06:33.080 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-16T15:06:33.080 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000430312 s, 1.2 MB/s 2024-09-16T15:06:33.081 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T15:06:33.130 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-09-16T15:06:33.177 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T15:06:33.177 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:06:33.177 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 894 Links: 1 2024-09-16T15:06:33.177 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:06:33.177 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-16 15:01:17.038979374 +0000 2024-09-16T15:06:33.177 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-16 15:01:16.798981518 +0000 2024-09-16T15:06:33.177 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-16 15:01:16.798981518 +0000 2024-09-16T15:06:33.177 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-09-16T15:06:33.178 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T15:06:33.231 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-16T15:06:33.232 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-16T15:06:33.232 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.00035514 s, 1.4 MB/s 2024-09-16T15:06:33.233 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T15:06:33.286 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-09-16T15:06:33.333 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T15:06:33.333 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:06:33.333 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-09-16T15:06:33.333 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:06:33.333 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-16 15:01:17.282977195 +0000 2024-09-16T15:06:33.334 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-16 15:01:17.278977231 +0000 2024-09-16T15:06:33.334 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-16 15:01:17.278977231 +0000 2024-09-16T15:06:33.334 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-09-16T15:06:33.334 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T15:06:33.387 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-16T15:06:33.388 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-16T15:06:33.388 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000453892 s, 1.1 MB/s 2024-09-16T15:06:33.389 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T15:06:33.439 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-09-16T15:06:33.485 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T15:06:33.485 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:06:33.485 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 930 Links: 1 2024-09-16T15:06:33.485 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:06:33.485 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-16 15:01:17.998970799 +0000 2024-09-16T15:06:33.485 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-16 15:01:17.750973015 +0000 2024-09-16T15:06:33.486 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-16 15:01:17.750973015 +0000 2024-09-16T15:06:33.486 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-09-16T15:06:33.486 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T15:06:33.539 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-16T15:06:33.540 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-16T15:06:33.540 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000399997 s, 1.3 MB/s 2024-09-16T15:06:33.541 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T15:06:33.590 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-16T15:06:33.590 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-09-16T15:06:33.590 INFO:tasks.ceph:Generating config... 2024-09-16T15:06:33.592 INFO:tasks.ceph:[client] debug client = 20 2024-09-16T15:06:33.592 INFO:tasks.ceph:[client] debug mgrc = 20 2024-09-16T15:06:33.592 INFO:tasks.ceph:[client] debug ms = 1 2024-09-16T15:06:33.592 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-16T15:06:33.592 INFO:tasks.ceph:[mds] debug mgrc = 20 2024-09-16T15:06:33.592 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-16T15:06:33.592 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T15:06:33.592 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T15:06:33.592 INFO:tasks.ceph:[mgr] mgr ttl cache expire seconds = 0 2024-09-16T15:06:33.592 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T15:06:33.593 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T15:06:33.593 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T15:06:33.593 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-16T15:06:33.593 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-16T15:06:33.593 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-16T15:06:33.593 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T15:06:33.593 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T15:06:33.593 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-16T15:06:33.593 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-16T15:06:33.594 INFO:tasks.ceph:[osd] debug mgrc = 20 2024-09-16T15:06:33.594 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T15:06:33.594 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T15:06:33.594 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-16T15:06:33.594 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T15:06:33.594 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T15:06:33.594 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T15:06:33.594 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T15:06:33.594 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T15:06:33.594 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T15:06:33.595 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T15:06:33.629 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T15:06:33.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T15:06:33.719 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:06:33.772 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.133'), ('mon.c', '[v2:172.21.15.133:3301,v1:172.21.15.133:6790]'), ('mon.b', '172.21.15.191')] 2024-09-16T15:06:33.773 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790],172.21.15.191', 'mon election default strategy': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'mgr ttl cache expire seconds': 0}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds': {'debug mgrc': 20}} 2024-09-16T15:06:33.774 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:33.774 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T15:06:33.817 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.133 --addv c '[v2:172.21.15.133:3301,v1:172.21.15.133:6790]' --add b 172.21.15.191 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:06:33.902 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:06:33.902 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: generated fsid 9e097bec-4223-42be-ab5d-08197519f9c7 2024-09-16T15:06:33.902 INFO:teuthology.orchestra.run.smithi133.stdout:setting min_mon_release = pacific 2024-09-16T15:06:33.902 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 0 2024-09-16T15:06:33.902 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 9e097bec-4223-42be-ab5d-08197519f9c7 2024-09-16T15:06:33.902 INFO:teuthology.orchestra.run.smithi133.stdout:last_changed 2024-09-16T15:06:33.903304+0000 2024-09-16T15:06:33.902 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-09-16T15:06:33.903304+0000 2024-09-16T15:06:33.902 INFO:teuthology.orchestra.run.smithi133.stdout:min_mon_release 16 (pacific) 2024-09-16T15:06:33.903 INFO:teuthology.orchestra.run.smithi133.stdout:election_strategy: 1 2024-09-16T15:06:33.903 INFO:teuthology.orchestra.run.smithi133.stdout:0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-09-16T15:06:33.903 INFO:teuthology.orchestra.run.smithi133.stdout:1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2024-09-16T15:06:33.903 INFO:teuthology.orchestra.run.smithi133.stdout:2: [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] mon.c 2024-09-16T15:06:33.903 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T15:06:33.904 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T15:06:33.950 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9e097bec-4223-42be-ab5d-08197519f9c7... 2024-09-16T15:06:33.951 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T15:06:33.994 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T15:06:34.022 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-09-16T15:06:34.022 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-09-16T15:06:34.022 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T15:06:34.022 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-09-16T15:06:34.023 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-09-16T15:06:34.024 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790],172.21.15.191" 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: mon election default strategy = 1 2024-09-16T15:06:34.025 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 9e097bec-4223-42be-ab5d-08197519f9c7 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.026 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2024-09-16T15:06:34.027 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 20 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 20 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgrc = 20 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 10 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.028 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: mgr ttl cache expire seconds = 0 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-16T15:06:34.029 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T15:06:34.030 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-09-16T15:06:34.031 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgrc = 20 2024-09-16T15:06:34.032 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-16T15:06:34.032 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-09-16T15:06:34.032 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-09-16T15:06:34.032 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-09-16T15:06:34.032 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-09-16T15:06:34.032 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-09-16T15:06:34.032 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-09-16T15:06:34.032 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-09-16T15:06:34.032 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgrc = 20 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.033 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2024-09-16T15:06:34.034 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: mon_allow_pool_size_one = true 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2024-09-16T15:06:34.035 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790],172.21.15.191" 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: mon election default strategy = 1 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = 9e097bec-4223-42be-ab5d-08197519f9c7 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: osd mclock profile = high_recovery_ops 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.036 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.037 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: bdev async discard = True 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: bdev enable discard = True 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore allocator = bitmap 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore block size = 96636764160 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore fsck on mount = True 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluefs = 20 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluestore = 20 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgrc = 20 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: debug rocksdb = 10 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd full ratio = 0.9 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd nearfull ratio = 0.8 2024-09-16T15:06:34.038 INFO:teuthology.orchestra.run.smithi191.stdout: osd failsafe full ratio = 0.95 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = bluestore 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: mgr ttl cache expire seconds = 0 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.039 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T15:06:34.040 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: # 1m isn't quite enough 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: mon_down_mkfs_grace = 2m 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_filestore_osds = false 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T15:06:34.041 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgrc = 20 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout:[mds] 2024-09-16T15:06:34.042 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgrc = 20 2024-09-16T15:06:34.043 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T15:06:34.043 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T15:06:34.133 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T15:06:34.133 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:34.133 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T15:06:34.138 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:34.139 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T15:06:34.186 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi133.front.sepia.ceph.com 2024-09-16T15:06:34.186 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:34.186 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T15:06:34.186 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:06:34.251 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:34.251 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:06:34.298 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi191.front.sepia.ceph.com 2024-09-16T15:06:34.298 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:34.298 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T15:06:34.298 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:06:34.321 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:34.321 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:06:34.369 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T15:06:34.370 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T15:06:34.370 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T15:06:34.413 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T15:06:34.416 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T15:06:34.458 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T15:06:34.476 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mgr/ceph-z && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.z /var/lib/ceph/mgr/ceph-z/keyring 2024-09-16T15:06:34.556 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mgr/ceph-z/keyring 2024-09-16T15:06:34.559 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T15:06:34.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-16T15:06:34.603 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-16T15:06:34.606 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-16T15:06:34.662 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-16T15:06:34.747 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-16T15:06:34.750 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-16T15:06:34.804 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-16T15:06:34.851 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-16T15:06:34.854 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-16T15:06:34.866 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T15:06:34.866 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T15:06:34.899 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T15:06:34.909 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T15:06:34.942 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-16T15:06:34.952 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T15:06:34.952 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-09-16T15:06:34.953 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T15:06:34.964 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-09-16T15:06:34.964 INFO:tasks.ceph:role: osd.0 2024-09-16T15:06:34.965 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-16T15:06:34.965 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T15:06:35.024 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:06:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:06:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:06:35.024 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T15:06:35.025 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:06:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:06:35.025 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:06:35.025 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T15:06:35.025 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:06:35.025 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:06:35.475 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-09-16T15:06:35.478 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-09-16T15:06:35.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T15:06:35.570 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T15:06:35.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:06:35.580 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T15:06:35.580 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:06:35.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:35.691+0000 7f946706f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T15:06:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:35.695+0000 7f946706f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T15:06:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:35.695+0000 7f946706f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T15:06:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:35.695+0000 7f946706f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T15:06:35.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:35.695+0000 7f946706f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T15:06:35.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:35.703+0000 7f946706f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T15:06:38.679 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T15:06:38.692 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T15:06:38.707 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-16T15:06:38.707 INFO:tasks.ceph:role: osd.1 2024-09-16T15:06:38.707 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-16T15:06:38.707 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T15:06:38.770 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:06:38.770 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:06:38.770 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:06:38.770 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T15:06:38.770 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:06:38.770 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:06:38.770 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:06:38.770 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T15:06:38.770 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:06:38.770 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:06:39.257 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-09-16T15:06:39.259 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-09-16T15:06:39.259 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-09-16T15:06:39.408 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T15:06:39.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:06:39.420 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T15:06:39.421 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T15:06:39.481 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-16T15:06:39.481 INFO:tasks.ceph:role: osd.2 2024-09-16T15:06:39.481 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-16T15:06:39.481 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T15:06:39.544 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:06:39.545 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:06:39.545 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:06:39.545 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T15:06:39.545 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:06:39.545 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:06:39.545 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:06:39.545 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T15:06:39.545 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:06:39.545 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:06:39.987 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-09-16T15:06:39.990 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-09-16T15:06:39.990 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2024-09-16T15:06:40.078 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T15:06:40.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:06:40.088 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T15:06:40.088 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T15:06:40.144 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-16T15:06:40.144 INFO:tasks.ceph:role: osd.3 2024-09-16T15:06:40.144 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-16T15:06:40.144 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T15:06:40.209 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:06:40.209 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:06:40.209 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:06:40.209 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-16T15:06:40.209 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:06:40.210 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:06:40.210 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:06:40.210 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-16T15:06:40.210 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:06:40.210 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:06:40.681 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2024-09-16T15:06:40.683 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2024-09-16T15:06:40.684 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2024-09-16T15:06:40.780 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T15:06:40.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:06:40.790 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: /sbin/restorecon: command not found 2024-09-16T15:06:40.790 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:06:40.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:40.893+0000 7f1600bf57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T15:06:40.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:40.893+0000 7f1600bf57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T15:06:40.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:40.893+0000 7f1600bf57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T15:06:40.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:40.893+0000 7f1600bf57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T15:06:40.900 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:40.893+0000 7f1600bf57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T15:06:40.906 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:40.905+0000 7f1600bf57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T15:06:43.872 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T15:06:43.885 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:06:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:43.965+0000 7f50d01ce7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T15:06:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:43.965+0000 7f50d01ce7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T15:06:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:43.965+0000 7f50d01ce7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T15:06:43.967 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:43.965+0000 7f50d01ce7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T15:06:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:43.965+0000 7f50d01ce7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T15:06:43.968 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:43.965+0000 7f50d01ce7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T15:06:46.972 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T15:06:47.033 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:06:47.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:47.117+0000 7f6b588b37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T15:06:47.118 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:47.117+0000 7f6b588b37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T15:06:47.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:47.117+0000 7f6b588b37c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T15:06:47.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:47.117+0000 7f6b588b37c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T15:06:47.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:47.117+0000 7f6b588b37c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T15:06:47.119 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:47.117+0000 7f6b588b37c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T15:06:50.096 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T15:06:50.149 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T15:06:50.150 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:50.150 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T15:06:50.163 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:50.163 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-16T15:06:50.217 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:50.217 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-16T15:06:50.273 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:50.273 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T15:06:50.329 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:50.329 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T15:06:50.342 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:50.343 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mgr/ceph-z/keyring of=/dev/stdout 2024-09-16T15:06:50.396 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:50.397 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-16T15:06:50.451 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:50.451 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T15:06:50.505 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:50.505 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T15:06:50.559 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:50.559 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T15:06:50.612 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:50.612 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T15:06:50.618 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:50.618 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-16T15:06:50.657 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T15:06:50.657 DEBUG:teuthology.orchestra.run.smithi133:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T15:06:50.663 DEBUG:teuthology.orchestra.run.smithi191:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T15:06:50.703 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.x] 2024-09-16T15:06:50.703 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQB6Sehm0TiwGBAA+EZnDByZ4BC+YHZ+VxpWjQ== 2024-09-16T15:06:50.703 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-09-16T15:06:50.703 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQB6SehmFnQFJBAA+aWCpQaB2wFvGdCdxZGeZg== 2024-09-16T15:06:50.703 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-09-16T15:06:50.703 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQB6SehmfsefLBAAzPPd8uZg0ecVpyzAvSKqfQ== 2024-09-16T15:06:50.703 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.0] 2024-09-16T15:06:50.703 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQB7Sehmv2xpKRAA++yyzQ2Ob1/Ab/rlRXW4rQ== 2024-09-16T15:06:50.703 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.y] 2024-09-16T15:06:50.703 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQB6SehmNXxTGxAA8gV4NMyJpKQgGhKSRjRDFQ== 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.z] 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQB6SehmfBgzIRAAOVimjG7pxagxcsQJRaKYPw== 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQB6Sehmo3jAMhAAMvk+GSsjXhdYzD4NNAPzwA== 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.1] 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCASehmSNqKNRAARiPpErEOWn8wPN9/5OnhGg== 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.2] 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCDSehm6m6oORAAVrndB7Tdv3FcGXCEeR74ng== 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.3] 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCHSehmHIARBxAAAJirCSzRMJAQATmqw4WVWw== 2024-09-16T15:06:50.704 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-09-16T15:06:50.705 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQB6SehmRkyuNRAAqyWa/zZAsEkR6rLaCFAlTQ== 2024-09-16T15:06:50.705 INFO:teuthology.orchestra.run.smithi133.stdout:[client.1] 2024-09-16T15:06:50.705 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQB6Sehmqk8xOBAAnco3G+7qcBqoKvjru6eSAA== 2024-09-16T15:06:50.711 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.x] 2024-09-16T15:06:50.711 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB6Sehm0TiwGBAA+EZnDByZ4BC+YHZ+VxpWjQ== 2024-09-16T15:06:50.711 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2024-09-16T15:06:50.711 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB6SehmFnQFJBAA+aWCpQaB2wFvGdCdxZGeZg== 2024-09-16T15:06:50.711 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2024-09-16T15:06:50.711 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB6SehmfsefLBAAzPPd8uZg0ecVpyzAvSKqfQ== 2024-09-16T15:06:50.711 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.0] 2024-09-16T15:06:50.711 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB7Sehmv2xpKRAA++yyzQ2Ob1/Ab/rlRXW4rQ== 2024-09-16T15:06:50.711 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.y] 2024-09-16T15:06:50.711 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB6SehmNXxTGxAA8gV4NMyJpKQgGhKSRjRDFQ== 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.z] 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB6SehmfBgzIRAAOVimjG7pxagxcsQJRaKYPw== 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB6Sehmo3jAMhAAMvk+GSsjXhdYzD4NNAPzwA== 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.1] 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCASehmSNqKNRAARiPpErEOWn8wPN9/5OnhGg== 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.2] 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCDSehm6m6oORAAVrndB7Tdv3FcGXCEeR74ng== 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.3] 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQCHSehmHIARBxAAAJirCSzRMJAQATmqw4WVWw== 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout:[client.0] 2024-09-16T15:06:50.712 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB6SehmRkyuNRAAqyWa/zZAsEkR6rLaCFAlTQ== 2024-09-16T15:06:50.713 INFO:teuthology.orchestra.run.smithi191.stdout:[client.1] 2024-09-16T15:06:50.713 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQB6Sehmqk8xOBAAnco3G+7qcBqoKvjru6eSAA== 2024-09-16T15:06:50.713 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:06:50.750 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:06:50.802 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:06:50.804 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:06:50.893 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:06:50.895 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:06:50.937 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:06:50.978 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:06:51.018 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:06:51.058 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:06:51.098 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.z --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:06:51.138 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.z --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:06:51.207 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:06:51.222 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:06:51.272 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:06:51.275 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:06:51.313 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:06:51.354 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:06:51.393 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:06:51.434 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:06:51.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:06:51.519 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:06:51.564 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:06:51.566 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:06:51.645 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T15:06:51.645 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T15:06:51.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:06:51.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T15:06:51.807 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T15:06:51.859 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:06:51.952 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T15:06:51.963 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T15:06:51.977 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:06:52.074 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T15:06:52.086 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:06:52.088 DEBUG:teuthology.orchestra.run.smithi191:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:06:52.136 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T15:06:52.136 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T15:06:52.137 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T15:06:52.139 INFO:tasks.ceph.mon.a:Started 2024-09-16T15:06:52.139 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T15:06:52.139 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T15:06:52.141 INFO:tasks.ceph.mon.c:Started 2024-09-16T15:06:52.141 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T15:06:52.142 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T15:06:52.182 INFO:tasks.ceph.mon.b:Started 2024-09-16T15:06:52.182 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T15:06:52.182 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T15:06:52.182 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T15:06:52.184 INFO:tasks.ceph.mgr.x:Started 2024-09-16T15:06:52.185 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T15:06:52.185 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T15:06:52.187 INFO:tasks.ceph.mgr.y:Started 2024-09-16T15:06:52.187 INFO:tasks.ceph.mgr.z:Restarting daemon 2024-09-16T15:06:52.187 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i z 2024-09-16T15:06:52.189 INFO:tasks.ceph.mgr.z:Started 2024-09-16T15:06:52.189 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T15:06:52.189 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T15:06:52.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d32a93640 1 Processor -- start 2024-09-16T15:06:52.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d32a93640 1 -- start start 2024-09-16T15:06:52.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d32a93640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d32a93640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2c0a5ed0 con 0x7f1d2c136720 2024-09-16T15:06:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d32a93640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2c135f50 con 0x7f1d2c0a84e0 2024-09-16T15:06:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d32a93640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2c136210 con 0x7f1d2c0a80e0 2024-09-16T15:06:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d3082f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1d2c136720 0x7f1d2c1395a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:41838/0 (socket says 172.21.15.133:41838) 2024-09-16T15:06:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d3082f640 1 -- 172.21.15.133:0/1639146109 learned_addr learned my addr 172.21.15.133:0/1639146109 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:06:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 msgr2=0x7f1d2c0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:06:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 msgr2=0x7f1d2c0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.303+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:06:52.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.507+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:52.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.507+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 msgr2=0x7f1d2c0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:06:52.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.507+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 msgr2=0x7f1d2c0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:52.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.507+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:52.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.507+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:06:52.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.911+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:52.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.911+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 msgr2=0x7f1d2c0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:06:52.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.911+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 msgr2=0x7f1d2c0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:52.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.911+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:52.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:52.911+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:06:53.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:53.715+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:53.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:53.715+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 msgr2=0x7f1d2c0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:06:53.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:53.715+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 msgr2=0x7f1d2c0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:53.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:53.715+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:53.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:53.715+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:06:55.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2a7fc640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 msgr2=0x7f1d2c0aac60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:06:55.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2a7fc640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.191:3300/0 conn(0x7f1d2c0a80e0 0x7f1d2c0aac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:55.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2a7fc640 1 -- 172.21.15.133:0/1639146109 >> v1:172.21.15.191:6789/0 conn(0x7f1d2c0a84e0 legacy=0x7f1d2c14dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2a7fc640 1 -- 172.21.15.133:0/1639146109 >> v1:172.21.15.133:6789/0 conn(0x7f1d2c136720 legacy=0x7f1d2c1395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2a7fc640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2a7fc640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2a7fc640 1 -- 172.21.15.133:0/1639146109 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2c136210 con 0x7f1d100036f0 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2a7fc640 1 -- 172.21.15.133:0/1639146109 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d10008d80 con 0x7f1d10006190 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 msgr2=0x7f1d10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 msgr2=0x7f1d10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d3082f640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 msgr2=0x7f1d10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:55.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d3082f640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 msgr2=0x7f1d10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:06:55.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.303+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:06:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.507+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.507+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.507+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 msgr2=0x7f1d10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:06:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.507+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 msgr2=0x7f1d10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.507+0000 7f1d3082f640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 msgr2=0x7f1d10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:06:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.507+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.507+0000 7f1d3082f640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 msgr2=0x7f1d10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.507+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.507+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:06:55.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.507+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:06:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.910+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.910+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.910+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 msgr2=0x7f1d10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:06:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.910+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 msgr2=0x7f1d10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.910+0000 7f1d3082f640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 msgr2=0x7f1d10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:06:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.910+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.910+0000 7f1d3082f640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 msgr2=0x7f1d10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.910+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.910+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:06:55.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:55.910+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:06:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:56.714+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:56.714+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:56.714+0000 7f1d3082f640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 msgr2=0x7f1d10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-16T15:06:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:56.714+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 msgr2=0x7f1d10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T15:06:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:56.714+0000 7f1d3082f640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 msgr2=0x7f1d10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:56.714+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 msgr2=0x7f1d10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T15:06:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:56.714+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:56.714+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:06:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:56.714+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:06:56.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:56.714+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T15:06:57.862 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:57.860+0000 7fd85f0dd180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T15:06:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2a7fc640 1 -- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 msgr2=0x7f1d10008650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:06:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2a7fc640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d10006190 0x7f1d10008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2a7fc640 1 -- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 msgr2=0x7f1d10005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2a7fc640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2a7fc640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d31030640 1 -- 172.21.15.133:0/1639146109 reap_dead start 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2a7fc640 1 -- 172.21.15.133:0/1639146109 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2c0a89c0 con 0x7f1d2c136720 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2a7fc640 1 -- 172.21.15.133:0/1639146109 --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d10011140 con 0x7f1d2c0a84e0 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2a7fc640 1 -- 172.21.15.133:0/1639146109 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2c136210 con 0x7f1d100036f0 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> v2:172.21.15.133:3300/0 conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> v1:172.21.15.191:6789/0 conn(0x7f1d2c0a84e0 legacy=0x7f1d1000b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 >> v1:172.21.15.133:6789/0 conn(0x7f1d2c136720 legacy=0x7f1d1000e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.302+0000 7f1d2bfff640 1 -- 172.21.15.133:0/1639146109 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d10007670 con 0x7f1d100036f0 2024-09-16T15:06:58.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 0x7f1d10005b50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d18000ee0 tx=0x7f1d1801dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcd5e6cd4b1d1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:58.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d2b7fe640 1 -- 172.21.15.133:0/1639146109 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d180a6d60 con 0x7f1d100036f0 2024-09-16T15:06:58.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d2b7fe640 1 -- 172.21.15.133:0/1639146109 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1800c040 con 0x7f1d100036f0 2024-09-16T15:06:58.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d2b7fe640 1 -- 172.21.15.133:0/1639146109 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d180ae680 con 0x7f1d100036f0 2024-09-16T15:06:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d32a93640 1 -- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 msgr2=0x7f1d10005b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d32a93640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 0x7f1d10005b50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1d18000ee0 tx=0x7f1d1801dd80 comp rx=0 tx=0).stop 2024-09-16T15:06:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d32a93640 1 -- 172.21.15.133:0/1639146109 shutdown_connections 2024-09-16T15:06:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d32a93640 1 --2- 172.21.15.133:0/1639146109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 0x7f1d10005b50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d32a93640 1 -- 172.21.15.133:0/1639146109 >> 172.21.15.133:0/1639146109 conn(0x7f1d2c06a260 msgr2=0x7f1d2c06bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d32a93640 1 -- 172.21.15.133:0/1639146109 shutdown_connections 2024-09-16T15:06:58.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d32a93640 1 -- 172.21.15.133:0/1639146109 wait complete. 2024-09-16T15:06:58.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d32a93640 1 Processor -- start 2024-09-16T15:06:58.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.306+0000 7f1d32a93640 1 -- start start 2024-09-16T15:06:58.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d32a93640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 0x7f1d2c1ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d32a93640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d2c0a84e0 0x7f1d2c1dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d32a93640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c136720 0x7f1d2c146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d32a93640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c1dc8a0 con 0x7f1d100036f0 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d32a93640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c1dcb60 con 0x7f1d2c0a84e0 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d32a93640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2c1cf470 con 0x7f1d2c136720 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d3082f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 0x7f1d2c1ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d31030640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c136720 0x7f1d2c146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d3082f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 0x7f1d2c1ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53240/0 (socket says 172.21.15.133:53240) 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d3082f640 1 -- 172.21.15.133:0/1082377951 learned_addr learned my addr 172.21.15.133:0/1082377951 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d2bfff640 1 --2- 172.21.15.133:0/1082377951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d2c0a84e0 0x7f1d2c1dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d3082f640 1 -- 172.21.15.133:0/1082377951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c136720 msgr2=0x7f1d2c146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1082377951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c136720 0x7f1d2c146ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d3082f640 1 -- 172.21.15.133:0/1082377951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d2c0a84e0 msgr2=0x7f1d2c1dbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1082377951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d2c0a84e0 0x7f1d2c1dbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d3082f640 1 -- 172.21.15.133:0/1082377951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1800e040 con 0x7f1d100036f0 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d3082f640 1 --2- 172.21.15.133:0/1082377951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 0x7f1d2c1ce740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1d200075f0 tx=0x7f1d20001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d297fa640 1 -- 172.21.15.133:0/1082377951 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2001c030 con 0x7f1d100036f0 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d297fa640 1 -- 172.21.15.133:0/1082377951 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d200077a0 con 0x7f1d100036f0 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d32a93640 1 -- 172.21.15.133:0/1082377951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d2c0b6c30 con 0x7f1d100036f0 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d297fa640 1 -- 172.21.15.133:0/1082377951 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d20004c40 con 0x7f1d100036f0 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d32a93640 1 -- 172.21.15.133:0/1082377951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d2c09fc90 con 0x7f1d100036f0 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d297fa640 1 -- 172.21.15.133:0/1082377951 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1d2001f070 con 0x7f1d100036f0 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d297fa640 20 mgrc handle_mgr_map mgrmap(e 1) v1 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d297fa640 4 mgrc handle_mgr_map Got map version 1 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d297fa640 4 mgrc handle_mgr_map Active mgr is now 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d297fa640 4 mgrc reconnect No active mgr available yet 2024-09-16T15:06:58.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d297fa640 1 -- 172.21.15.133:0/1082377951 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1d2001d070 con 0x7f1d100036f0 2024-09-16T15:06:58.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.310+0000 7f1d32a93640 1 -- 172.21.15.133:0/1082377951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2c09f0c0 con 0x7f1d100036f0 2024-09-16T15:06:58.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.314+0000 7f1d297fa640 1 -- 172.21.15.133:0/1082377951 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f1d2c09f0c0 con 0x7f1d100036f0 2024-09-16T15:06:58.372 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:58.368+0000 7fe2f950a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T15:06:58.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.382+0000 7f1d32a93640 1 -- 172.21.15.133:0/1082377951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1d2c139960 con 0x7f1d100036f0 2024-09-16T15:06:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.386+0000 7f1d297fa640 1 -- 172.21.15.133:0/1082377951 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1d20004e40 con 0x7f1d100036f0 2024-09-16T15:06:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:adjusted tunables profile to default 2024-09-16T15:06:58.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.390+0000 7f1d32a93640 10 mgrc shutdown 2024-09-16T15:06:58.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.390+0000 7f1d32a93640 1 -- 172.21.15.133:0/1082377951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 msgr2=0x7f1d2c1ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.390+0000 7f1d32a93640 1 --2- 172.21.15.133:0/1082377951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 0x7f1d2c1ce740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1d200075f0 tx=0x7f1d20001640 comp rx=0 tx=0).stop 2024-09-16T15:06:58.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.390+0000 7f1d32a93640 1 -- 172.21.15.133:0/1082377951 shutdown_connections 2024-09-16T15:06:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.390+0000 7f1d32a93640 1 --2- 172.21.15.133:0/1082377951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c136720 0x7f1d2c146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.390+0000 7f1d32a93640 1 --2- 172.21.15.133:0/1082377951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d2c0a84e0 0x7f1d2c1dbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.390+0000 7f1d32a93640 1 --2- 172.21.15.133:0/1082377951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d100036f0 0x7f1d2c1ce740 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.390+0000 7f1d32a93640 1 -- 172.21.15.133:0/1082377951 >> 172.21.15.133:0/1082377951 conn(0x7f1d2c06a260 msgr2=0x7f1d2c071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.390+0000 7f1d32a93640 1 -- 172.21.15.133:0/1082377951 shutdown_connections 2024-09-16T15:06:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:58.390+0000 7f1d32a93640 1 -- 172.21.15.133:0/1082377951 wait complete. 2024-09-16T15:06:58.404 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:58.400+0000 7fd85f0dd180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T15:06:58.413 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T15:06:58.413 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T15:06:58.413 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:06:58.413 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T15:06:58.423 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:58.423 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T15:06:58.436 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:58.436 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T15:06:58.495 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-16T15:06:58.495 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T15:06:58.551 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab 0 2024-09-16T15:06:58.552 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:06:58.546+0000 7fee80690180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T15:06:58.615 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:58.612+0000 7fd85f0dd180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T15:06:58.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d36852640 1 Processor -- start 2024-09-16T15:06:58.717 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d36852640 1 -- start start 2024-09-16T15:06:58.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d36852640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7d3009f490 0x7f7d30140780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d36852640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d30143f60 con 0x7f7d300a7540 2024-09-16T15:06:58.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d36852640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d30144220 con 0x7f7d300a7920 2024-09-16T15:06:58.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d36852640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d301444e0 con 0x7f7d3009f490 2024-09-16T15:06:58.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d2f7fe640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7d3009f490 0x7f7d30140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d2f7fe640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7d3009f490 0x7f7d30140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:52928/0 (socket says 172.21.15.191:52928) 2024-09-16T15:06:58.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d2f7fe640 1 -- 172.21.15.191:0/366620786 learned_addr learned my addr 172.21.15.191:0/366620786 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-16T15:06:58.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d2f7fe640 1 -- 172.21.15.191:0/366620786 >> v1:172.21.15.191:6789/0 conn(0x7f7d300a7920 legacy=0x7f7d3014a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d2f7fe640 1 -- 172.21.15.191:0/366620786 >> v1:172.21.15.133:6789/0 conn(0x7f7d300a7540 legacy=0x7f7d30143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:58.719 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.716+0000 7f7d2f7fe640 1 -- 172.21.15.191:0/366620786 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d30143ca0 con 0x7f7d3009f490 2024-09-16T15:06:58.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d2f7fe640 1 --2- 172.21.15.191:0/366620786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 0x7f7d30140780 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400bb10 tx=0x7f7d2401eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f7f541142d5b0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:58.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d2effd640 1 -- 172.21.15.191:0/366620786 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240a7b30 con 0x7f7d3009f490 2024-09-16T15:06:58.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d2effd640 1 -- 172.21.15.191:0/366620786 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d240a7c90 con 0x7f7d3009f490 2024-09-16T15:06:58.722 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d2effd640 1 -- 172.21.15.191:0/366620786 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240af440 con 0x7f7d3009f490 2024-09-16T15:06:58.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d36852640 1 -- 172.21.15.191:0/366620786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 msgr2=0x7f7d30140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d36852640 1 --2- 172.21.15.191:0/366620786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 0x7f7d30140780 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400bb10 tx=0x7f7d2401eb40 comp rx=0 tx=0).stop 2024-09-16T15:06:58.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d36852640 1 -- 172.21.15.191:0/366620786 shutdown_connections 2024-09-16T15:06:58.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d36852640 1 --2- 172.21.15.191:0/366620786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 0x7f7d30140780 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d36852640 1 -- 172.21.15.191:0/366620786 >> 172.21.15.191:0/366620786 conn(0x7f7d300a4f70 msgr2=0x7f7d300a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:58.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d36852640 1 -- 172.21.15.191:0/366620786 shutdown_connections 2024-09-16T15:06:58.723 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d36852640 1 -- 172.21.15.191:0/366620786 wait complete. 2024-09-16T15:06:58.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d36852640 1 Processor -- start 2024-09-16T15:06:58.724 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.720+0000 7f7d36852640 1 -- start start 2024-09-16T15:06:58.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d36852640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 0x7f7d300a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d36852640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d300a7540 0x7f7d300b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d36852640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d300a7920 0x7f7d301dbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d36852640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301db820 con 0x7f7d300a7540 2024-09-16T15:06:58.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d36852640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d301db990 con 0x7f7d3009f490 2024-09-16T15:06:58.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d36852640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d300b6970 con 0x7f7d300a7920 2024-09-16T15:06:58.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 0x7f7d300a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.726 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 0x7f7d300a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:52938/0 (socket says 172.21.15.191:52938) 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2ffff640 1 -- 172.21.15.191:0/3657544947 learned_addr learned my addr 172.21.15.191:0/3657544947 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2f7fe640 1 --2- 172.21.15.191:0/3657544947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d300a7540 0x7f7d300b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d34def640 1 --2- 172.21.15.191:0/3657544947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d300a7920 0x7f7d301dbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2ffff640 1 -- 172.21.15.191:0/3657544947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d300a7920 msgr2=0x7f7d301dbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2ffff640 1 --2- 172.21.15.191:0/3657544947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d300a7920 0x7f7d301dbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2ffff640 1 -- 172.21.15.191:0/3657544947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d300a7540 msgr2=0x7f7d300b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2ffff640 1 --2- 172.21.15.191:0/3657544947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d300a7540 0x7f7d300b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2ffff640 1 -- 172.21.15.191:0/3657544947 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2400b770 con 0x7f7d3009f490 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2ffff640 1 --2- 172.21.15.191:0/3657544947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 0x7f7d300a21d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800c450 tx=0x7f7d2800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2cff9640 1 -- 172.21.15.191:0/3657544947 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28012830 con 0x7f7d3009f490 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2cff9640 1 -- 172.21.15.191:0/3657544947 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d280129b0 con 0x7f7d3009f490 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d36852640 1 -- 172.21.15.191:0/3657544947 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d30071ed0 con 0x7f7d3009f490 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2cff9640 1 -- 172.21.15.191:0/3657544947 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d28012d60 con 0x7f7d3009f490 2024-09-16T15:06:58.727 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d36852640 1 -- 172.21.15.191:0/3657544947 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d300b3ca0 con 0x7f7d3009f490 2024-09-16T15:06:58.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2cff9640 1 -- 172.21.15.191:0/3657544947 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f7d2801c760 con 0x7f7d3009f490 2024-09-16T15:06:58.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2cff9640 20 mgrc handle_mgr_map mgrmap(e 1) v1 2024-09-16T15:06:58.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2cff9640 4 mgrc handle_mgr_map Got map version 1 2024-09-16T15:06:58.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2cff9640 4 mgrc handle_mgr_map Active mgr is now 2024-09-16T15:06:58.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2cff9640 4 mgrc reconnect No active mgr available yet 2024-09-16T15:06:58.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d2cff9640 1 -- 172.21.15.191:0/3657544947 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7d2801b850 con 0x7f7d3009f490 2024-09-16T15:06:58.728 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.724+0000 7f7d36852640 1 -- 172.21.15.191:0/3657544947 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3006aa50 con 0x7f7d3009f490 2024-09-16T15:06:58.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.740+0000 7f7d2cff9640 1 -- 172.21.15.191:0/3657544947 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f7d2801ba60 con 0x7f7d3009f490 2024-09-16T15:06:58.778 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:58.776+0000 7fe2f950a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T15:06:58.811 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.808+0000 7f7d36852640 1 -- 172.21.15.191:0/3657544947 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab", "id": 0} v 0) v1 -- 0x7f7d300b5240 con 0x7f7d3009f490 2024-09-16T15:06:58.818 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d2cff9640 1 -- 172.21.15.191:0/3657544947 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7d2801bc00 con 0x7f7d3009f490 2024-09-16T15:06:58.818 INFO:teuthology.orchestra.run.smithi191.stdout:0 2024-09-16T15:06:58.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d36852640 10 mgrc shutdown 2024-09-16T15:06:58.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d36852640 1 -- 172.21.15.191:0/3657544947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 msgr2=0x7f7d300a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d36852640 1 --2- 172.21.15.191:0/3657544947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 0x7f7d300a21d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800c450 tx=0x7f7d2800c910 comp rx=0 tx=0).stop 2024-09-16T15:06:58.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d36852640 1 -- 172.21.15.191:0/3657544947 shutdown_connections 2024-09-16T15:06:58.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d36852640 1 --2- 172.21.15.191:0/3657544947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d300a7920 0x7f7d301dbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d36852640 1 --2- 172.21.15.191:0/3657544947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d300a7540 0x7f7d300b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d36852640 1 --2- 172.21.15.191:0/3657544947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d3009f490 0x7f7d300a21d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d36852640 1 -- 172.21.15.191:0/3657544947 >> 172.21.15.191:0/3657544947 conn(0x7f7d300a4f70 msgr2=0x7f7d3006c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:58.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d36852640 1 -- 172.21.15.191:0/3657544947 shutdown_connections 2024-09-16T15:06:58.820 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.816+0000 7f7d36852640 1 -- 172.21.15.191:0/3657544947 wait complete. 2024-09-16T15:06:58.835 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 193fc20e-147a-4af2-9254-9239013f2784 1 2024-09-16T15:06:58.914 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:58.912+0000 7fe2f950a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T15:06:58.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.988+0000 7f3d91b7c640 1 Processor -- start 2024-09-16T15:06:58.992 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.988+0000 7f3d91b7c640 1 -- start start 2024-09-16T15:06:58.993 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d91b7c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a4090 0x7f3d8c06cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.993 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d91b7c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8c06a4a0 con 0x7f3d8c0a37e0 2024-09-16T15:06:58.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d91b7c640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d8c0a6c50 con 0x7f3d8c06a0c0 2024-09-16T15:06:58.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d91b7c640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0a44e0 con 0x7f3d8c0a4090 2024-09-16T15:06:58.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8b7fe640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f3d8c06a0c0 0x7f3d8c143f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:54608/0 (socket says 172.21.15.191:54608) 2024-09-16T15:06:58.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8b7fe640 1 -- 172.21.15.191:0/2184945252 learned_addr learned my addr 172.21.15.191:0/2184945252 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-09-16T15:06:58.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8bfff640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a4090 0x7f3d8c06cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767641021 0 0) 0x7f3d8c0a6c50 con 0x7f3d8c06a0c0 2024-09-16T15:06:58.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d74003530 con 0x7f3d8c06a0c0 2024-09-16T15:06:58.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1913883973 0 0) 0x7f3d74003530 con 0x7f3d8c06a0c0 2024-09-16T15:06:58.995 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a4090 msgr2=0x7f3d8c06cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a4090 0x7f3d8c06cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 >> v1:172.21.15.133:6789/0 conn(0x7f3d8c0a37e0 legacy=0x7f3d8c14bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c144610 con 0x7f3d8c06a0c0 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f3d7c001680 con 0x7f3d8c06a0c0 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 >> v1:172.21.15.191:6789/0 conn(0x7f3d8c06a0c0 legacy=0x7f3d8c143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d74003980 0x7f3d74003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d740047d0 0x7f3d74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a4090 0x7f3d74005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7400ba60 con 0x7f3d74003980 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7400bbd0 con 0x7f3d740047d0 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7400bd90 con 0x7f3d8c0a4090 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8affd640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d740047d0 0x7f3d74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8b7fe640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d74003980 0x7f3d74003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:58.996 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8affd640 1 -- 172.21.15.191:0/2184945252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a4090 msgr2=0x7f3d74005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:58.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8affd640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a4090 0x7f3d74005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8affd640 1 -- 172.21.15.191:0/2184945252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d74003980 msgr2=0x7f3d74003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:58.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8affd640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d74003980 0x7f3d74003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:58.997 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.992+0000 7f3d8affd640 1 -- 172.21.15.191:0/2184945252 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d74003530 con 0x7f3d740047d0 2024-09-16T15:06:58.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d8affd640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d740047d0 0x7f3d74004c50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001780 tx=0x7f3d8001c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8f074e50155c416 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:58.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d800052d0 con 0x7f3d740047d0 2024-09-16T15:06:58.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d800a7050 con 0x7f3d740047d0 2024-09-16T15:06:58.998 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d8a7fc640 1 -- 172.21.15.191:0/2184945252 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80003600 con 0x7f3d740047d0 2024-09-16T15:06:58.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d91b7c640 1 -- 172.21.15.191:0/2184945252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d740047d0 msgr2=0x7f3d74004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:58.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d91b7c640 1 --2- 172.21.15.191:0/2184945252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d740047d0 0x7f3d74004c50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001780 tx=0x7f3d8001c400 comp rx=0 tx=0).stop 2024-09-16T15:06:58.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d8bfff640 1 -- 172.21.15.191:0/2184945252 reap_dead start 2024-09-16T15:06:58.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d91b7c640 1 -- 172.21.15.191:0/2184945252 shutdown_connections 2024-09-16T15:06:58.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d91b7c640 1 -- 172.21.15.191:0/2184945252 >> 172.21.15.191:0/2184945252 conn(0x7f3d8c0a51b0 msgr2=0x7f3d8c0a55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:58.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d91b7c640 1 -- 172.21.15.191:0/2184945252 shutdown_connections 2024-09-16T15:06:58.999 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d91b7c640 1 -- 172.21.15.191:0/2184945252 wait complete. 2024-09-16T15:06:59.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d91b7c640 1 Processor -- start 2024-09-16T15:06:59.000 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d91b7c640 1 -- start start 2024-09-16T15:06:59.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:58.996+0000 7f3d91b7c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d740047d0 0x7f3d8c13e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d91b7c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a0e60 0x7f3d8c0a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d91b7c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d8c145960 0x7f3d8c145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d91b7c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0b3a90 con 0x7f3d740047d0 2024-09-16T15:06:59.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d91b7c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c0a1660 con 0x7f3d8c145960 2024-09-16T15:06:59.002 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d91b7c640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c13c330 con 0x7f3d8c0a0e60 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8bfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d8c145960 0x7f3d8c145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8bfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d8c145960 0x7f3d8c145df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:52950/0 (socket says 172.21.15.191:52950) 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8bfff640 1 -- 172.21.15.191:0/1250557340 learned_addr learned my addr 172.21.15.191:0/1250557340 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8affd640 1 --2- 172.21.15.191:0/1250557340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a0e60 0x7f3d8c0a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8b7fe640 1 --2- 172.21.15.191:0/1250557340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d740047d0 0x7f3d8c13e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8bfff640 1 -- 172.21.15.191:0/1250557340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a0e60 msgr2=0x7f3d8c0a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8bfff640 1 --2- 172.21.15.191:0/1250557340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a0e60 0x7f3d8c0a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8bfff640 1 -- 172.21.15.191:0/1250557340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d740047d0 msgr2=0x7f3d8c13e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8bfff640 1 --2- 172.21.15.191:0/1250557340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d740047d0 0x7f3d8c13e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8bfff640 1 -- 172.21.15.191:0/1250557340 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c071910 con 0x7f3d8c145960 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d8bfff640 1 --2- 172.21.15.191:0/1250557340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d8c145960 0x7f3d8c145df0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c06abe0 tx=0x7f3d8400b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d73fff640 1 -- 172.21.15.191:0/1250557340 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84010040 con 0x7f3d8c145960 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d73fff640 1 -- 172.21.15.191:0/1250557340 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8400bcb0 con 0x7f3d8c145960 2024-09-16T15:06:59.003 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d91b7c640 1 -- 172.21.15.191:0/1250557340 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8c071bd0 con 0x7f3d8c145960 2024-09-16T15:06:59.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d73fff640 1 -- 172.21.15.191:0/1250557340 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84014640 con 0x7f3d8c145960 2024-09-16T15:06:59.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d91b7c640 1 -- 172.21.15.191:0/1250557340 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8c0a5870 con 0x7f3d8c145960 2024-09-16T15:06:59.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d73fff640 1 -- 172.21.15.191:0/1250557340 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f3d84014aa0 con 0x7f3d8c145960 2024-09-16T15:06:59.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d73fff640 20 mgrc handle_mgr_map mgrmap(e 1) v1 2024-09-16T15:06:59.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d73fff640 4 mgrc handle_mgr_map Got map version 1 2024-09-16T15:06:59.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d73fff640 4 mgrc handle_mgr_map Active mgr is now 2024-09-16T15:06:59.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d73fff640 4 mgrc reconnect No active mgr available yet 2024-09-16T15:06:59.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d73fff640 1 -- 172.21.15.191:0/1250557340 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f3d8401dd80 con 0x7f3d8c145960 2024-09-16T15:06:59.004 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d91b7c640 1 -- 172.21.15.191:0/1250557340 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8c143f00 con 0x7f3d8c145960 2024-09-16T15:06:59.005 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.000+0000 7f3d73fff640 1 -- 172.21.15.191:0/1250557340 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f3d84018020 con 0x7f3d8c145960 2024-09-16T15:06:59.072 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.072+0000 7f3d91b7c640 1 -- 172.21.15.191:0/1250557340 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "193fc20e-147a-4af2-9254-9239013f2784", "id": 1} v 0) v1 -- 0x7f3d8c06d440 con 0x7f3d8c145960 2024-09-16T15:06:59.077 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:06:59.074+0000 7fee80690180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T15:06:59.079 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.076+0000 7f3d73fff640 1 -- 172.21.15.191:0/1250557340 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "193fc20e-147a-4af2-9254-9239013f2784", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3d8402c0b0 con 0x7f3d8c145960 2024-09-16T15:06:59.079 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-09-16T15:06:59.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.076+0000 7f3d91b7c640 10 mgrc shutdown 2024-09-16T15:06:59.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.076+0000 7f3d91b7c640 1 -- 172.21.15.191:0/1250557340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d8c145960 msgr2=0x7f3d8c145df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.080 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.076+0000 7f3d91b7c640 1 --2- 172.21.15.191:0/1250557340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d8c145960 0x7f3d8c145df0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c06abe0 tx=0x7f3d8400b480 comp rx=0 tx=0).stop 2024-09-16T15:06:59.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.076+0000 7f3d91b7c640 1 -- 172.21.15.191:0/1250557340 shutdown_connections 2024-09-16T15:06:59.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.076+0000 7f3d91b7c640 1 --2- 172.21.15.191:0/1250557340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3d8c145960 0x7f3d8c145df0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.076+0000 7f3d91b7c640 1 --2- 172.21.15.191:0/1250557340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d8c0a0e60 0x7f3d8c0a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.076+0000 7f3d91b7c640 1 --2- 172.21.15.191:0/1250557340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d740047d0 0x7f3d8c13e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.076+0000 7f3d91b7c640 1 -- 172.21.15.191:0/1250557340 >> 172.21.15.191:0/1250557340 conn(0x7f3d8c0a51b0 msgr2=0x7f3d8c09ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:59.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.076+0000 7f3d91b7c640 1 -- 172.21.15.191:0/1250557340 shutdown_connections 2024-09-16T15:06:59.081 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.080+0000 7f3d91b7c640 1 -- 172.21.15.191:0/1250557340 wait complete. 2024-09-16T15:06:59.097 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 10507e21-41d0-4f66-ada7-6fb5a2837b77 2 2024-09-16T15:06:59.198 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:59.196+0000 7fd85f0dd180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T15:06:59.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad55706640 1 Processor -- start 2024-09-16T15:06:59.250 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad55706640 1 -- start start 2024-09-16T15:06:59.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad55706640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad50137950 0x7fad500a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad55706640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fad500a9400 0x7fad500a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad55706640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad50133b10 con 0x7fad50134220 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad55706640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad50133d40 con 0x7fad50137950 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad55706640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad50134000 con 0x7fad500a9400 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad4f7fe640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fad500a9400 0x7fad500a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad4f7fe640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fad500a9400 0x7fad500a78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:52964/0 (socket says 172.21.15.191:52964) 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad4effd640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fad50134220 0x7fad500a66c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:54616/0 (socket says 172.21.15.191:54616) 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad4f7fe640 1 -- 172.21.15.191:0/1529832374 learned_addr learned my addr 172.21.15.191:0/1529832374 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad4e7fc640 1 --2- 172.21.15.191:0/1529832374 >> v2:172.21.15.133:3300/0 conn(0x7fad50137950 0x7fad500a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad4f7fe640 1 -- 172.21.15.191:0/1529832374 >> v2:172.21.15.133:3300/0 conn(0x7fad50137950 msgr2=0x7fad500a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad4f7fe640 1 --2- 172.21.15.191:0/1529832374 >> v2:172.21.15.133:3300/0 conn(0x7fad50137950 0x7fad500a8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad4f7fe640 1 -- 172.21.15.191:0/1529832374 >> v1:172.21.15.191:6789/0 conn(0x7fad50134220 legacy=0x7fad500a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad4dffb640 1 -- 172.21.15.191:0/1529832374 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885508103 0 0) 0x7fad50133b10 con 0x7fad50134220 2024-09-16T15:06:59.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.248+0000 7fad4f7fe640 1 -- 172.21.15.191:0/1529832374 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad500a7e00 con 0x7fad500a9400 2024-09-16T15:06:59.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad4f7fe640 1 --2- 172.21.15.191:0/1529832374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad500a9400 0x7fad500a78c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fad48009e50 tx=0x7fad4801c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29bde7b0be74f8a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:59.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad4dffb640 1 -- 172.21.15.191:0/1529832374 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480a5650 con 0x7fad500a9400 2024-09-16T15:06:59.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad4dffb640 1 -- 172.21.15.191:0/1529832374 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad480a57b0 con 0x7fad500a9400 2024-09-16T15:06:59.254 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad4dffb640 1 -- 172.21.15.191:0/1529832374 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480a5b40 con 0x7fad500a9400 2024-09-16T15:06:59.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad55706640 1 -- 172.21.15.191:0/1529832374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad500a9400 msgr2=0x7fad500a78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad55706640 1 --2- 172.21.15.191:0/1529832374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad500a9400 0x7fad500a78c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fad48009e50 tx=0x7fad4801c880 comp rx=0 tx=0).stop 2024-09-16T15:06:59.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad55706640 1 -- 172.21.15.191:0/1529832374 shutdown_connections 2024-09-16T15:06:59.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad55706640 1 --2- 172.21.15.191:0/1529832374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad500a9400 0x7fad500a78c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad55706640 1 --2- 172.21.15.191:0/1529832374 >> v2:172.21.15.133:3300/0 conn(0x7fad50137950 0x7fad500a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad55706640 1 -- 172.21.15.191:0/1529832374 >> 172.21.15.191:0/1529832374 conn(0x7fad5006f000 msgr2=0x7fad5006f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:59.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad55706640 1 -- 172.21.15.191:0/1529832374 shutdown_connections 2024-09-16T15:06:59.255 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad55706640 1 -- 172.21.15.191:0/1529832374 wait complete. 2024-09-16T15:06:59.256 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fd85f0dd180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T15:06:59.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad55706640 1 Processor -- start 2024-09-16T15:06:59.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.252+0000 7fad55706640 1 -- start start 2024-09-16T15:06:59.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad55706640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad500a9400 0x7fad5014ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad55706640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad50134220 0x7fad5009af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad55706640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad50137950 0x7fad5009b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad55706640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500a3b30 con 0x7fad50134220 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad55706640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5009cd70 con 0x7fad50137950 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad55706640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5014b9b0 con 0x7fad500a9400 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad50137950 0x7fad5009b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad50137950 0x7fad5009b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:52966/0 (socket says 172.21.15.191:52966) 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4f7fe640 1 -- 172.21.15.191:0/1587507637 learned_addr learned my addr 172.21.15.191:0/1587507637 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4e7fc640 1 --2- 172.21.15.191:0/1587507637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad50134220 0x7fad5009af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4effd640 1 --2- 172.21.15.191:0/1587507637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad500a9400 0x7fad5014ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4f7fe640 1 -- 172.21.15.191:0/1587507637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad500a9400 msgr2=0x7fad5014ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4f7fe640 1 --2- 172.21.15.191:0/1587507637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad500a9400 0x7fad5014ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4f7fe640 1 -- 172.21.15.191:0/1587507637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad50134220 msgr2=0x7fad5009af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4f7fe640 1 --2- 172.21.15.191:0/1587507637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad50134220 0x7fad5009af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4f7fe640 1 -- 172.21.15.191:0/1587507637 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad48009b30 con 0x7fad50137950 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4effd640 1 --2- 172.21.15.191:0/1587507637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad500a9400 0x7fad5014ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad4f7fe640 1 --2- 172.21.15.191:0/1587507637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad50137950 0x7fad5009b4c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fad48019490 tx=0x7fad48019620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad377fe640 1 -- 172.21.15.191:0/1587507637 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480a5650 con 0x7fad50137950 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad377fe640 1 -- 172.21.15.191:0/1587507637 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad48007bb0 con 0x7fad50137950 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad55706640 1 -- 172.21.15.191:0/1587507637 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5009fed0 con 0x7fad50137950 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad377fe640 1 -- 172.21.15.191:0/1587507637 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48009510 con 0x7fad50137950 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad55706640 1 -- 172.21.15.191:0/1587507637 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5009cab0 con 0x7fad50137950 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad377fe640 1 -- 172.21.15.191:0/1587507637 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fad480b8770 con 0x7fad50137950 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad377fe640 20 mgrc handle_mgr_map mgrmap(e 1) v1 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad377fe640 4 mgrc handle_mgr_map Got map version 1 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad377fe640 4 mgrc handle_mgr_map Active mgr is now 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad377fe640 4 mgrc reconnect No active mgr available yet 2024-09-16T15:06:59.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad377fe640 1 -- 172.21.15.191:0/1587507637 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fad480c1af0 con 0x7fad50137950 2024-09-16T15:06:59.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.256+0000 7fad55706640 1 -- 172.21.15.191:0/1587507637 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad500672a0 con 0x7fad50137950 2024-09-16T15:06:59.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.260+0000 7fad377fe640 1 -- 172.21.15.191:0/1587507637 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fad480bc020 con 0x7fad50137950 2024-09-16T15:06:59.289 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:06:59.282+0000 7fee80690180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T15:06:59.312 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:59.312+0000 7fd85f0dd180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T15:06:59.328 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.328+0000 7fad55706640 1 -- 172.21.15.191:0/1587507637 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "10507e21-41d0-4f66-ada7-6fb5a2837b77", "id": 2} v 0) v1 -- 0x7fad500a4d60 con 0x7fad50137950 2024-09-16T15:06:59.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.332+0000 7fad377fe640 1 -- 172.21.15.191:0/1587507637 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "10507e21-41d0-4f66-ada7-6fb5a2837b77", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fad480c1d50 con 0x7fad50137950 2024-09-16T15:06:59.335 INFO:teuthology.orchestra.run.smithi191.stdout:2 2024-09-16T15:06:59.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.332+0000 7fad55706640 10 mgrc shutdown 2024-09-16T15:06:59.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.332+0000 7fad55706640 1 -- 172.21.15.191:0/1587507637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad50137950 msgr2=0x7fad5009b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.332+0000 7fad55706640 1 --2- 172.21.15.191:0/1587507637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad50137950 0x7fad5009b4c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fad48019490 tx=0x7fad48019620 comp rx=0 tx=0).stop 2024-09-16T15:06:59.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.336+0000 7fad55706640 1 -- 172.21.15.191:0/1587507637 shutdown_connections 2024-09-16T15:06:59.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.336+0000 7fad55706640 1 --2- 172.21.15.191:0/1587507637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad50137950 0x7fad5009b4c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.336+0000 7fad55706640 1 --2- 172.21.15.191:0/1587507637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad50134220 0x7fad5009af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.336+0000 7fad55706640 1 --2- 172.21.15.191:0/1587507637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad500a9400 0x7fad5014ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.336+0000 7fad55706640 1 -- 172.21.15.191:0/1587507637 >> 172.21.15.191:0/1587507637 conn(0x7fad5006f000 msgr2=0x7fad50069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:59.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.336+0000 7fad55706640 1 -- 172.21.15.191:0/1587507637 shutdown_connections 2024-09-16T15:06:59.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.336+0000 7fad55706640 1 -- 172.21.15.191:0/1587507637 wait complete. 2024-09-16T15:06:59.353 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new d4b8a403-1961-4a29-af2b-cfa07ce5f1bd 3 2024-09-16T15:06:59.355 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:59.352+0000 7fe2f950a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T15:06:59.395 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:59.392+0000 7fd85f0dd180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T15:06:59.410 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:59.408+0000 7fe2f950a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T15:06:59.451 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:59.448+0000 7fd85f0dd180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T15:06:59.461 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:59.460+0000 7fe2f950a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T15:06:59.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.460+0000 7f298031e640 1 Processor -- start 2024-09-16T15:06:59.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.460+0000 7f298031e640 1 -- start start 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f298031e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f297806a0c0 0x7f297809ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f298031e640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809f210 0x7f297806cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f298031e640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297806a4a0 con 0x7f297806d100 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f298031e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29780a2960 con 0x7f297806a0c0 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f298031e640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297809fa90 con 0x7f297809f210 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297e8bb640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f297806d100 0x7f297806b9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:54622/0 (socket says 172.21.15.191:54622) 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297e8bb640 1 -- 172.21.15.191:0/1256580739 learned_addr learned my addr 172.21.15.191:0/1256580739 (peer_addr_for_me v1:172.21.15.191:0/0) 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d8b9640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809f210 0x7f297806cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297e0ba640 1 --2- 172.21.15.191:0/1256580739 >> v2:172.21.15.133:3300/0 conn(0x7f297806a0c0 0x7f297809ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542136531 0 0) 0x7f297806a4a0 con 0x7f297806d100 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2964003530 con 0x7f297806d100 2024-09-16T15:06:59.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075448344 0 0) 0x7f2964003530 con 0x7f297806d100 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809f210 msgr2=0x7f297806cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809f210 0x7f297806cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 >> v2:172.21.15.133:3300/0 conn(0x7f297806a0c0 msgr2=0x7f297809ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 --2- 172.21.15.191:0/1256580739 >> v2:172.21.15.133:3300/0 conn(0x7f297806a0c0 0x7f297809ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297809f860 con 0x7f297806d100 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f2970001a00 con 0x7f297806d100 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 >> v1:172.21.15.191:6789/0 conn(0x7f297806d100 legacy=0x7f297806b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809f210 0x7f2964003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 0x7f29640047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2964004d10 0x7f2964005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29780a2960 con 0x7f2964004d10 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297809fa90 con 0x7f2964003f00 2024-09-16T15:06:59.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296400b990 con 0x7f297809f210 2024-09-16T15:06:59.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d8b9640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 0x7f29640047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297e0ba640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809f210 0x7f2964003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297e8bb640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2964004d10 0x7f2964005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d8b9640 1 -- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809f210 msgr2=0x7f2964003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:59.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d8b9640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809f210 0x7f2964003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d8b9640 1 -- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2964004d10 msgr2=0x7f2964005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:59.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d8b9640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2964004d10 0x7f2964005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.464+0000 7f297d8b9640 1 -- 172.21.15.191:0/1256580739 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2964003530 con 0x7f2964003f00 2024-09-16T15:06:59.470 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f297d8b9640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 0x7f29640047d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2968009ae0 tx=0x7f296801c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=598bf071fe7f3967 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:59.470 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680a5640 con 0x7f2964003f00 2024-09-16T15:06:59.470 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29680a57a0 con 0x7f2964003f00 2024-09-16T15:06:59.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f297d0b8640 1 -- 172.21.15.191:0/1256580739 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680a5b50 con 0x7f2964003f00 2024-09-16T15:06:59.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 -- 172.21.15.191:0/1256580739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 msgr2=0x7f29640047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 --2- 172.21.15.191:0/1256580739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 0x7f29640047d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2968009ae0 tx=0x7f296801c9f0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f297e8bb640 1 -- 172.21.15.191:0/1256580739 reap_dead start 2024-09-16T15:06:59.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 -- 172.21.15.191:0/1256580739 shutdown_connections 2024-09-16T15:06:59.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 -- 172.21.15.191:0/1256580739 >> 172.21.15.191:0/1256580739 conn(0x7f29780a51b0 msgr2=0x7f29780a55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:59.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 -- 172.21.15.191:0/1256580739 shutdown_connections 2024-09-16T15:06:59.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 -- 172.21.15.191:0/1256580739 wait complete. 2024-09-16T15:06:59.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 Processor -- start 2024-09-16T15:06:59.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 -- start start 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 0x7f29781cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809c120 0x7f297809cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f297809d240 0x7f297809a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297809dc50 con 0x7f297809d240 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297809d990 con 0x7f2964003f00 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f298031e640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297809d6c0 con 0x7f297809c120 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f297e0ba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 0x7f29781cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f297e0ba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 0x7f29781cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:52982/0 (socket says 172.21.15.191:52982) 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.468+0000 7f297e0ba640 1 -- 172.21.15.191:0/952689737 learned_addr learned my addr 172.21.15.191:0/952689737 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f297e8bb640 1 --2- 172.21.15.191:0/952689737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f297809d240 0x7f297809a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f297d8b9640 1 --2- 172.21.15.191:0/952689737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809c120 0x7f297809cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f297e0ba640 1 -- 172.21.15.191:0/952689737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809c120 msgr2=0x7f297809cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f297e0ba640 1 --2- 172.21.15.191:0/952689737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809c120 0x7f297809cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f297e0ba640 1 -- 172.21.15.191:0/952689737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f297809d240 msgr2=0x7f297809a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f297e0ba640 1 --2- 172.21.15.191:0/952689737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f297809d240 0x7f297809a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f297e0ba640 1 -- 172.21.15.191:0/952689737 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297809b440 con 0x7f2964003f00 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f297d8b9640 1 --2- 172.21.15.191:0/952689737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809c120 0x7f297809cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f297e0ba640 1 --2- 172.21.15.191:0/952689737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 0x7f29781cc6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2974011660 tx=0x7f2974011b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f296effd640 1 -- 172.21.15.191:0/952689737 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297400f9c0 con 0x7f2964003f00 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f296effd640 1 -- 172.21.15.191:0/952689737 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297400fb20 con 0x7f2964003f00 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f298031e640 1 -- 172.21.15.191:0/952689737 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29781d5ef0 con 0x7f2964003f00 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f296effd640 1 -- 172.21.15.191:0/952689737 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297400fe90 con 0x7f2964003f00 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f298031e640 1 -- 172.21.15.191:0/952689737 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297809a9a0 con 0x7f2964003f00 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f296effd640 1 -- 172.21.15.191:0/952689737 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f29740059a0 con 0x7f2964003f00 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f296effd640 20 mgrc handle_mgr_map mgrmap(e 1) v1 2024-09-16T15:06:59.475 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f296effd640 4 mgrc handle_mgr_map Got map version 1 2024-09-16T15:06:59.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f296effd640 4 mgrc handle_mgr_map Active mgr is now 2024-09-16T15:06:59.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f296effd640 4 mgrc reconnect No active mgr available yet 2024-09-16T15:06:59.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f296effd640 1 -- 172.21.15.191:0/952689737 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f2974006610 con 0x7f2964003f00 2024-09-16T15:06:59.476 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.472+0000 7f298031e640 1 -- 172.21.15.191:0/952689737 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297809ecd0 con 0x7f2964003f00 2024-09-16T15:06:59.477 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.476+0000 7f296effd640 1 -- 172.21.15.191:0/952689737 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f297403be30 con 0x7f2964003f00 2024-09-16T15:06:59.534 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:59.532+0000 7fd85f0dd180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T15:06:59.544 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:59.544+0000 7fe2f950a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T15:06:59.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.544+0000 7f298031e640 1 -- 172.21.15.191:0/952689737 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d4b8a403-1961-4a29-af2b-cfa07ce5f1bd", "id": 3} v 0) v1 -- 0x7f297809c5c0 con 0x7f2964003f00 2024-09-16T15:06:59.552 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.548+0000 7f296effd640 1 -- 172.21.15.191:0/952689737 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4b8a403-1961-4a29-af2b-cfa07ce5f1bd", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f297402a070 con 0x7f2964003f00 2024-09-16T15:06:59.552 INFO:teuthology.orchestra.run.smithi191.stdout:3 2024-09-16T15:06:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.552+0000 7f298031e640 10 mgrc shutdown 2024-09-16T15:06:59.553 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.552+0000 7f298031e640 1 -- 172.21.15.191:0/952689737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 msgr2=0x7f29781cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.552+0000 7f298031e640 1 --2- 172.21.15.191:0/952689737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 0x7f29781cc6b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2974011660 tx=0x7f2974011b20 comp rx=0 tx=0).stop 2024-09-16T15:06:59.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.552+0000 7f298031e640 1 -- 172.21.15.191:0/952689737 shutdown_connections 2024-09-16T15:06:59.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.552+0000 7f298031e640 1 --2- 172.21.15.191:0/952689737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f297809d240 0x7f297809a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.552+0000 7f298031e640 1 --2- 172.21.15.191:0/952689737 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f297809c120 0x7f297809cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.552+0000 7f298031e640 1 --2- 172.21.15.191:0/952689737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2964003f00 0x7f29781cc6b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.552+0000 7f298031e640 1 -- 172.21.15.191:0/952689737 >> 172.21.15.191:0/952689737 conn(0x7f29780a51b0 msgr2=0x7f29780a0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:59.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.552+0000 7f298031e640 1 -- 172.21.15.191:0/952689737 shutdown_connections 2024-09-16T15:06:59.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-16T15:06:59.552+0000 7f298031e640 1 -- 172.21.15.191:0/952689737 wait complete. 2024-09-16T15:06:59.569 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T15:06:59.569 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T15:06:59.572 INFO:tasks.ceph.osd.0:Started 2024-09-16T15:06:59.572 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T15:06:59.572 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T15:06:59.589 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:59.588+0000 7fd85f0dd180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T15:06:59.590 INFO:tasks.ceph.osd.1:Started 2024-09-16T15:06:59.590 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T15:06:59.590 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T15:06:59.592 INFO:tasks.ceph.osd.2:Started 2024-09-16T15:06:59.592 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T15:06:59.592 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T15:06:59.594 INFO:tasks.ceph.osd.3:Started 2024-09-16T15:06:59.594 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:06:59.598 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:59.596+0000 7fe2f950a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T15:06:59.697 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:59.696+0000 7fe2f950a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T15:06:59.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6db2dc640 1 Processor -- start 2024-09-16T15:06:59.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6db2dc640 1 -- start start 2024-09-16T15:06:59.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6db2dc640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff6d406ca50 0x7ff6d414f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6db2dc640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d40a41e0 con 0x7ff6d406b050 2024-09-16T15:06:59.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6db2dc640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d4146dd0 con 0x7ff6d406b430 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6db2dc640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d4147070 con 0x7ff6d406ca50 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9078640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff6d406b050 0x7ff6d41408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42688/0 (socket says 172.21.15.133:42688) 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9078640 1 -- 172.21.15.133:0/3395541154 learned_addr learned my addr 172.21.15.133:0/3395541154 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9879640 1 --2- 172.21.15.133:0/3395541154 >> v2:172.21.15.191:3300/0 conn(0x7ff6d406ca50 0x7ff6d414f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660662531 0 0) 0x7ff6d40a41e0 con 0x7ff6d406b050 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6c0003530 con 0x7ff6d406b050 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297871440 0 0) 0x7ff6d4146dd0 con 0x7ff6d406b430 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d40a41e0 con 0x7ff6d406b430 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1983575201 0 0) 0x7ff6c0003530 con 0x7ff6d406b050 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 >> v2:172.21.15.191:3300/0 conn(0x7ff6d406ca50 msgr2=0x7ff6d414f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 --2- 172.21.15.133:0/3395541154 >> v2:172.21.15.191:3300/0 conn(0x7ff6d406ca50 0x7ff6d414f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 >> v1:172.21.15.191:6789/0 conn(0x7ff6d406b430 legacy=0x7ff6d414bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4147330 con 0x7ff6d406b050 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7ff6d0001680 con 0x7ff6d406b050 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 >> v1:172.21.15.133:6789/0 conn(0x7ff6d406b050 legacy=0x7ff6d41408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c0003b10 0x7ff6c0003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c00047d0 0x7ff6c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6c0005190 0x7ff6c000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4147070 con 0x7ff6c00047d0 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c000c0c0 con 0x7ff6c0003b10 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c000c280 con 0x7ff6c0005190 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9078640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c0003b10 0x7ff6c0003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9879640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6c0005190 0x7ff6c000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d8877640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c00047d0 0x7ff6c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9879640 1 -- 172.21.15.133:0/3395541154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c0003b10 msgr2=0x7ff6c0003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:59.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9879640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c0003b10 0x7ff6c0003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9879640 1 -- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c00047d0 msgr2=0x7ff6c0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:06:59.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9879640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c00047d0 0x7ff6c0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9879640 1 -- 172.21.15.133:0/3395541154 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c0003530 con 0x7ff6c0005190 2024-09-16T15:06:59.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d9078640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6c0003b10 0x7ff6c0003f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:06:59.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.706+0000 7ff6d8877640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c00047d0 0x7ff6c0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:06:59.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9879640 1 -- 172.21.15.133:0/3395541154 reap_dead start 2024-09-16T15:06:59.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9879640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6c0005190 0x7ff6c000b990 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009fc0 tx=0x7ff6cc01cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76263a2936d1cdef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:59.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0a5c30 con 0x7ff6c0005190 2024-09-16T15:06:59.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc00f040 con 0x7ff6c0005190 2024-09-16T15:06:59.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6cbfff640 1 -- 172.21.15.133:0/3395541154 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc0039a0 con 0x7ff6c0005190 2024-09-16T15:06:59.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6c0005190 msgr2=0x7ff6c000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6c0005190 0x7ff6c000b990 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009fc0 tx=0x7ff6cc01cca0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- 172.21.15.133:0/3395541154 shutdown_connections 2024-09-16T15:06:59.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 --2- 172.21.15.133:0/3395541154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6c0005190 0x7ff6c000b990 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- 172.21.15.133:0/3395541154 >> 172.21.15.133:0/3395541154 conn(0x7ff6d406f000 msgr2=0x7ff6d406f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:59.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- 172.21.15.133:0/3395541154 shutdown_connections 2024-09-16T15:06:59.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- 172.21.15.133:0/3395541154 wait complete. 2024-09-16T15:06:59.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 Processor -- start 2024-09-16T15:06:59.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- start start 2024-09-16T15:06:59.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c0005190 0x7ff6d4143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6d4143ff0 0x7ff6d40b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d40b6550 0x7ff6d4142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:06:59.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d40b6ca0 con 0x7ff6c0005190 2024-09-16T15:06:59.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d40b6f60 con 0x7ff6d40b6550 2024-09-16T15:06:59.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4144470 con 0x7ff6d4143ff0 2024-09-16T15:06:59.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d8877640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6d4143ff0 0x7ff6d40b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9078640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c0005190 0x7ff6d4143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d8877640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6d4143ff0 0x7ff6d40b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:42784/0 (socket says 172.21.15.133:42784) 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d8877640 1 -- 172.21.15.133:0/2921834277 learned_addr learned my addr 172.21.15.133:0/2921834277 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9078640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c0005190 0x7ff6d4143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53254/0 (socket says 172.21.15.133:53254) 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9879640 1 --2- 172.21.15.133:0/2921834277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d40b6550 0x7ff6d4142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9078640 1 -- 172.21.15.133:0/2921834277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6d4143ff0 msgr2=0x7ff6d40b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9078640 1 --2- 172.21.15.133:0/2921834277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6d4143ff0 0x7ff6d40b6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9078640 1 -- 172.21.15.133:0/2921834277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d40b6550 msgr2=0x7ff6d4142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9078640 1 --2- 172.21.15.133:0/2921834277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d40b6550 0x7ff6d4142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9078640 1 -- 172.21.15.133:0/2921834277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d40b69d0 con 0x7ff6c0005190 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d8877640 1 --2- 172.21.15.133:0/2921834277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6d4143ff0 0x7ff6d40b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6d9078640 1 --2- 172.21.15.133:0/2921834277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c0005190 0x7ff6d4143ab0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0007310 tx=0x7ff6d0007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6c9ffb640 1 -- 172.21.15.133:0/2921834277 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0007e30 con 0x7ff6c0005190 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6c9ffb640 1 -- 172.21.15.133:0/2921834277 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d000f960 con 0x7ff6c0005190 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- 172.21.15.133:0/2921834277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d41432d0 con 0x7ff6c0005190 2024-09-16T15:06:59.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6c9ffb640 1 -- 172.21.15.133:0/2921834277 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000fcd0 con 0x7ff6c0005190 2024-09-16T15:06:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6db2dc640 1 -- 172.21.15.133:0/2921834277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d4138a40 con 0x7ff6c0005190 2024-09-16T15:06:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6c9ffb640 1 -- 172.21.15.133:0/2921834277 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff6d001d760 con 0x7ff6c0005190 2024-09-16T15:06:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6c9ffb640 20 mgrc handle_mgr_map mgrmap(e 1) v1 2024-09-16T15:06:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6c9ffb640 4 mgrc handle_mgr_map Got map version 1 2024-09-16T15:06:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6c9ffb640 4 mgrc handle_mgr_map Active mgr is now 2024-09-16T15:06:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6c9ffb640 4 mgrc reconnect No active mgr available yet 2024-09-16T15:06:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.710+0000 7ff6c9ffb640 1 -- 172.21.15.133:0/2921834277 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff6d0013b30 con 0x7ff6c0005190 2024-09-16T15:06:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.714+0000 7ff6db2dc640 1 -- 172.21.15.133:0/2921834277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d40672a0 con 0x7ff6c0005190 2024-09-16T15:06:59.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.714+0000 7ff6c9ffb640 1 -- 172.21.15.133:0/2921834277 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7ff6d000fe30 con 0x7ff6c0005190 2024-09-16T15:06:59.747 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:59.744+0000 7fe2f950a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T15:06:59.761 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:59.760+0000 7fd85f0dd180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T15:06:59.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 1 -- 172.21.15.133:0/2921834277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6d40a8940 con 0x7ff6c0005190 2024-09-16T15:06:59.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6c9ffb640 1 -- 172.21.15.133:0/2921834277 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4152 (secure 0 0 0) 0x7ff6d002c090 con 0x7ff6c0005190 2024-09-16T15:06:59.778 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:06:59.778 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":6,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:06:59.549440+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:06:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 10 mgrc shutdown 2024-09-16T15:06:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 1 -- 172.21.15.133:0/2921834277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c0005190 msgr2=0x7ff6d4143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:06:59.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 1 --2- 172.21.15.133:0/2921834277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c0005190 0x7ff6d4143ab0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0007310 tx=0x7ff6d0007710 comp rx=0 tx=0).stop 2024-09-16T15:06:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 1 -- 172.21.15.133:0/2921834277 shutdown_connections 2024-09-16T15:06:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 1 --2- 172.21.15.133:0/2921834277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6d40b6550 0x7ff6d4142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 1 --2- 172.21.15.133:0/2921834277 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff6d4143ff0 0x7ff6d40b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 1 --2- 172.21.15.133:0/2921834277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6c0005190 0x7ff6d4143ab0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:06:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 1 -- 172.21.15.133:0/2921834277 >> 172.21.15.133:0/2921834277 conn(0x7ff6d406f000 msgr2=0x7ff6d4144dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:06:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 1 -- 172.21.15.133:0/2921834277 shutdown_connections 2024-09-16T15:06:59.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:06:59.774+0000 7ff6db2dc640 1 -- 172.21.15.133:0/2921834277 wait complete. 2024-09-16T15:06:59.795 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T15:06:59.795 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T15:06:59.821 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:06:59.820+0000 7fd85f0dd180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T15:06:59.852 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:06:59.846+0000 7fee80690180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T15:06:59.887 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:59.884+0000 7fe2f950a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T15:06:59.906 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:06:59.902+0000 7fee80690180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T15:06:59.938 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:06:59.936+0000 7fe2f950a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T15:06:59.959 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:06:59.954+0000 7fee80690180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T15:07:00.037 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:00.034+0000 7fee80690180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T15:07:00.089 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:00.086+0000 7fee80690180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T15:07:00.166 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:00.162+0000 7fee80690180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T15:07:00.191 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-16T15:07:00.186+0000 7ff268cad7c0 -1 Falling back to public interface 2024-09-16T15:07:00.198 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:00.205 INFO:tasks.ceph.osd.2.smithi191.stderr:2024-09-16T15:07:00.204+0000 7fcbf967d7c0 -1 Falling back to public interface 2024-09-16T15:07:00.206 INFO:tasks.ceph.osd.2.smithi191.stderr:2024-09-16T15:07:00.204+0000 7fcbf863e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-09-16T15:07:00.206 INFO:tasks.ceph.osd.3.smithi191.stderr:2024-09-16T15:07:00.204+0000 7fd4155427c0 -1 Falling back to public interface 2024-09-16T15:07:00.213 INFO:tasks.ceph.osd.1.smithi191.stderr:2024-09-16T15:07:00.212+0000 7f104d8467c0 -1 Falling back to public interface 2024-09-16T15:07:00.219 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:00.214+0000 7fee80690180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T15:07:00.304 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f9374e50640 1 Processor -- start 2024-09-16T15:07:00.304 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f9374e50640 1 -- start start 2024-09-16T15:07:00.305 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f9374e50640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f93700a7810 0x7f93700a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:00.305 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f9374e50640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f9370140c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:00.305 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f9374e50640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93700a8590 con 0x7f937013f770 2024-09-16T15:07:00.305 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f9374e50640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93700a4bf0 con 0x7f93700a7810 2024-09-16T15:07:00.305 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f9374e50640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93700a3920 con 0x7f93701408a0 2024-09-16T15:07:00.306 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f936dd9b640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f9370140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:00.306 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f936dd9b640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f9370140c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:42794/0 (socket says 172.21.15.133:42794) 2024-09-16T15:07:00.306 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f936ed9d640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f937013f770 0x7f937013fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42696/0 (socket says 172.21.15.133:42696) 2024-09-16T15:07:00.306 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f936dd9b640 1 -- 172.21.15.133:0/1860079999 learned_addr learned my addr 172.21.15.133:0/1860079999 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:00.306 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.298+0000 7f936e59c640 1 --2- 172.21.15.133:0/1860079999 >> v2:172.21.15.191:3300/0 conn(0x7f93700a7810 0x7f93700a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:00.306 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f936dd9b640 1 -- 172.21.15.133:0/1860079999 >> v2:172.21.15.191:3300/0 conn(0x7f93700a7810 msgr2=0x7f93700a7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:00.306 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f936dd9b640 1 --2- 172.21.15.133:0/1860079999 >> v2:172.21.15.191:3300/0 conn(0x7f93700a7810 0x7f93700a7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:00.306 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f936d59a640 1 -- 172.21.15.133:0/1860079999 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516999204 0 0) 0x7f93700a8590 con 0x7f937013f770 2024-09-16T15:07:00.306 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f936dd9b640 1 -- 172.21.15.133:0/1860079999 >> v1:172.21.15.133:6789/0 conn(0x7f937013f770 legacy=0x7f937013fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:00.306 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f936dd9b640 1 -- 172.21.15.133:0/1860079999 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937009f5e0 con 0x7f93701408a0 2024-09-16T15:07:00.307 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f936dd9b640 1 --2- 172.21.15.133:0/1860079999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f9370140c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9364009870 tx=0x7f936401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c357e0b66f7a698 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:00.307 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f936d59a640 1 -- 172.21.15.133:0/1860079999 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93640ad640 con 0x7f93701408a0 2024-09-16T15:07:00.307 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f936d59a640 1 -- 172.21.15.133:0/1860079999 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93640ad7a0 con 0x7f93701408a0 2024-09-16T15:07:00.307 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f936d59a640 1 -- 172.21.15.133:0/1860079999 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93640adb30 con 0x7f93701408a0 2024-09-16T15:07:00.308 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f9374e50640 1 -- 172.21.15.133:0/1860079999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 msgr2=0x7f9370140c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:00.308 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f9374e50640 1 --2- 172.21.15.133:0/1860079999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f9370140c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9364009870 tx=0x7f936401c540 comp rx=0 tx=0).stop 2024-09-16T15:07:00.308 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f9374e50640 1 -- 172.21.15.133:0/1860079999 shutdown_connections 2024-09-16T15:07:00.308 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f9374e50640 1 --2- 172.21.15.133:0/1860079999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f9370140c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:00.309 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f9374e50640 1 --2- 172.21.15.133:0/1860079999 >> v2:172.21.15.191:3300/0 conn(0x7f93700a7810 0x7f93700a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:00.309 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f9374e50640 1 -- 172.21.15.133:0/1860079999 >> 172.21.15.133:0/1860079999 conn(0x7f93700a4f70 msgr2=0x7f93700a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:00.309 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f9374e50640 1 -- 172.21.15.133:0/1860079999 shutdown_connections 2024-09-16T15:07:00.309 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.302+0000 7f9374e50640 1 -- 172.21.15.133:0/1860079999 wait complete. 2024-09-16T15:07:00.311 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 Processor -- start 2024-09-16T15:07:00.311 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 -- start start 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93700a7810 0x7f9370145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937013f770 0x7f93700b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f93700b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700b5110 con 0x7f937013f770 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93700b5ab0 con 0x7f93700a7810 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93700b4e50 con 0x7f93701408a0 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936ed9d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f93700b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937013f770 0x7f93700b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936ed9d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f93700b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:42810/0 (socket says 172.21.15.133:42810) 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937013f770 0x7f93700b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53262/0 (socket says 172.21.15.133:53262) 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936ed9d640 1 -- 172.21.15.133:0/2426791600 learned_addr learned my addr 172.21.15.133:0/2426791600 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:00.312 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936e59c640 1 --2- 172.21.15.133:0/2426791600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93700a7810 0x7f9370145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936dd9b640 1 -- 172.21.15.133:0/2426791600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 msgr2=0x7f93700b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936dd9b640 1 --2- 172.21.15.133:0/2426791600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f93700b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936dd9b640 1 -- 172.21.15.133:0/2426791600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93700a7810 msgr2=0x7f9370145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936dd9b640 1 --2- 172.21.15.133:0/2426791600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93700a7810 0x7f9370145500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936dd9b640 1 -- 172.21.15.133:0/2426791600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93640094d0 con 0x7f937013f770 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936ed9d640 1 --2- 172.21.15.133:0/2426791600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f93700b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f936dd9b640 1 --2- 172.21.15.133:0/2426791600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937013f770 0x7f93700b3a20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f93640ac000 tx=0x7f93640192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f935effd640 1 -- 172.21.15.133:0/2426791600 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93640b6a30 con 0x7f937013f770 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f935effd640 1 -- 172.21.15.133:0/2426791600 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93640ba070 con 0x7f937013f770 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 -- 172.21.15.133:0/2426791600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9370145e50 con 0x7f937013f770 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f935effd640 1 -- 172.21.15.133:0/2426791600 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93640b6e10 con 0x7f937013f770 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 -- 172.21.15.133:0/2426791600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93700b54f0 con 0x7f937013f770 2024-09-16T15:07:00.313 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f9374e50640 1 -- 172.21.15.133:0/2426791600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9370146330 con 0x7f937013f770 2024-09-16T15:07:00.314 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f935effd640 1 -- 172.21.15.133:0/2426791600 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f93640c9750 con 0x7f937013f770 2024-09-16T15:07:00.314 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f935effd640 20 mgrc handle_mgr_map mgrmap(e 1) v1 2024-09-16T15:07:00.314 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f935effd640 4 mgrc handle_mgr_map Got map version 1 2024-09-16T15:07:00.314 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f935effd640 4 mgrc handle_mgr_map Active mgr is now 2024-09-16T15:07:00.314 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f935effd640 4 mgrc reconnect No active mgr available yet 2024-09-16T15:07:00.314 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.306+0000 7f935effd640 1 -- 172.21.15.133:0/2426791600 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f93640c0910 con 0x7f937013f770 2024-09-16T15:07:00.314 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.310+0000 7f935effd640 1 -- 172.21.15.133:0/2426791600 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f93640bc070 con 0x7f937013f770 2024-09-16T15:07:00.371 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 1 -- 172.21.15.133:0/2426791600 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93700a7bf0 con 0x7f937013f770 2024-09-16T15:07:00.371 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f935effd640 1 -- 172.21.15.133:0/2426791600 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4152 (secure 0 0 0) 0x7f93640b2070 con 0x7f937013f770 2024-09-16T15:07:00.371 INFO:teuthology.misc.health.smithi133.stdout: 2024-09-16T15:07:00.372 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":6,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:06:59.549440+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:00.373 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 10 mgrc shutdown 2024-09-16T15:07:00.373 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 1 -- 172.21.15.133:0/2426791600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937013f770 msgr2=0x7f93700b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:00.373 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 1 --2- 172.21.15.133:0/2426791600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937013f770 0x7f93700b3a20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f93640ac000 tx=0x7f93640192e0 comp rx=0 tx=0).stop 2024-09-16T15:07:00.373 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 1 -- 172.21.15.133:0/2426791600 shutdown_connections 2024-09-16T15:07:00.373 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 1 --2- 172.21.15.133:0/2426791600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93701408a0 0x7f93700b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:00.373 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 1 --2- 172.21.15.133:0/2426791600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937013f770 0x7f93700b3a20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:00.373 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 1 --2- 172.21.15.133:0/2426791600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93700a7810 0x7f9370145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:00.373 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 1 -- 172.21.15.133:0/2426791600 >> 172.21.15.133:0/2426791600 conn(0x7f93700a4f70 msgr2=0x7f93700a1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:00.373 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 1 -- 172.21.15.133:0/2426791600 shutdown_connections 2024-09-16T15:07:00.373 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:00.366+0000 7f9374e50640 1 -- 172.21.15.133:0/2426791600 wait complete. 2024-09-16T15:07:00.376 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:00.370+0000 7fee80690180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T15:07:00.386 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-16T15:07:00.437 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:00.434+0000 7fee80690180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T15:07:00.500 INFO:tasks.ceph.mgr.y.smithi191.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T15:07:00.500 INFO:tasks.ceph.mgr.y.smithi191.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T15:07:00.500 INFO:tasks.ceph.mgr.y.smithi191.stderr: from numpy import show_config as show_numpy_config 2024-09-16T15:07:00.500 INFO:tasks.ceph.mgr.z.smithi191.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T15:07:00.500 INFO:tasks.ceph.mgr.z.smithi191.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T15:07:00.500 INFO:tasks.ceph.mgr.z.smithi191.stderr: from numpy import show_config as show_numpy_config 2024-09-16T15:07:00.549 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:00.548+0000 7fd85f0dd180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T15:07:00.550 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:00.548+0000 7fe2f950a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T15:07:00.694 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:00.692+0000 7fd85f0dd180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:07:00.700 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:00.696+0000 7fe2f950a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:07:00.851 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:00.848+0000 7fd85f0dd180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T15:07:00.852 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:00.848+0000 7fe2f950a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T15:07:00.915 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:00.912+0000 7fe2f950a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T15:07:00.915 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:00.912+0000 7fd85f0dd180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T15:07:01.049 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:01.048+0000 7fe2f950a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:07:01.050 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:01.048+0000 7fd85f0dd180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:07:01.096 INFO:tasks.ceph.mgr.x.smithi133.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T15:07:01.096 INFO:tasks.ceph.mgr.x.smithi133.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T15:07:01.096 INFO:tasks.ceph.mgr.x.smithi133.stderr: from numpy import show_config as show_numpy_config 2024-09-16T15:07:01.164 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:01.158+0000 7fee80690180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T15:07:01.347 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:01.342+0000 7fee80690180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:07:01.518 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:01.514+0000 7fee80690180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T15:07:01.589 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:01.586+0000 7fee80690180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T15:07:01.737 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:01.730+0000 7fee80690180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:07:01.984 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:01.980+0000 7fd85f0dd180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T15:07:01.985 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:01.984+0000 7fe2f950a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T15:07:02.061 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:02.060+0000 7fd85f0dd180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T15:07:02.061 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:02.060+0000 7fe2f950a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T15:07:02.244 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:02.240+0000 7fd85f0dd180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T15:07:02.244 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:02.240+0000 7fe2f950a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T15:07:02.254 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:07:02.257 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:07:02.319 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:02.316+0000 7fd85f0dd180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T15:07:02.320 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:02.316+0000 7fe2f950a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T15:07:02.351 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:02.346+0000 7fee80690180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T15:07:02.427 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:02.422+0000 7fee80690180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T15:07:02.609 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:02.602+0000 7fee80690180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T15:07:02.611 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:02.608+0000 7fd85f0dd180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T15:07:02.611 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:02.608+0000 7fe2f950a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T15:07:02.674 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:07:02.672+0000 7fe2f950a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T15:07:02.674 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:02.672+0000 7fd85f0dd180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T15:07:02.685 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:02.682+0000 7fee80690180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T15:07:02.987 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:02.982+0000 7fee80690180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T15:07:03.057 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:07:03.050+0000 7fee80690180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T15:07:03.711 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:03.708+0000 7fd806692640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T15:07:03.712 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:07:03.712+0000 7fd806692640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T15:07:03.734 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-16T15:07:03.730+0000 7ff268cad7c0 -1 osd.0 0 log_to_monitors true 2024-09-16T15:07:04.074 INFO:tasks.ceph.osd.2.smithi191.stderr:2024-09-16T15:07:04.072+0000 7fcbf967d7c0 -1 osd.2 0 log_to_monitors true 2024-09-16T15:07:04.089 INFO:tasks.ceph.osd.1.smithi191.stderr:2024-09-16T15:07:04.088+0000 7f104d8467c0 -1 osd.1 0 log_to_monitors true 2024-09-16T15:07:04.166 INFO:tasks.ceph.osd.3.smithi191.stderr:2024-09-16T15:07:04.164+0000 7fd4155427c0 -1 osd.3 0 log_to_monitors true 2024-09-16T15:07:06.546 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-16T15:07:06.542+0000 7ff26442f640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T15:07:06.558 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-09-16T15:07:06.554+0000 7ff25ca74640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:07:06.790 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:06.909 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1f2b0640 1 Processor -- start 2024-09-16T15:07:06.909 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1f2b0640 1 -- start start 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1f2b0640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a5000 0x7feb180a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1f2b0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb180a8a40 con 0x7feb180a7de0 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1f2b0640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb180b8010 con 0x7feb180a6df0 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1f2b0640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb180a8840 con 0x7feb180a5000 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1d04c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7feb180a7de0 0x7feb180b71f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42704/0 (socket says 172.21.15.133:42704) 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1d04c640 1 -- 172.21.15.133:0/830560452 learned_addr learned my addr 172.21.15.133:0/830560452 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1d84d640 1 --2- 172.21.15.133:0/830560452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a5000 0x7feb180a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb0ffff640 1 -- 172.21.15.133:0/830560452 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443176096 0 0) 0x7feb180a8a40 con 0x7feb180a7de0 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb0ffff640 1 -- 172.21.15.133:0/830560452 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb04003530 con 0x7feb180a7de0 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1d84d640 1 -- 172.21.15.133:0/830560452 >> v1:172.21.15.191:6789/0 conn(0x7feb180a6df0 legacy=0x7feb180b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1d84d640 1 -- 172.21.15.133:0/830560452 >> v1:172.21.15.133:6789/0 conn(0x7feb180a7de0 legacy=0x7feb180b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:06.911 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.906+0000 7feb1d84d640 1 -- 172.21.15.133:0/830560452 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb180a9620 con 0x7feb180a5000 2024-09-16T15:07:06.913 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d84d640 1 --2- 172.21.15.133:0/830560452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a5000 0x7feb180a8f40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feb10009dc0 tx=0x7feb1001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=662834ec1ab6864 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:06.913 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb0ffff640 1 -- 172.21.15.133:0/830560452 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100a59c0 con 0x7feb180a5000 2024-09-16T15:07:06.913 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb0ffff640 1 -- 172.21.15.133:0/830560452 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb100a5b20 con 0x7feb180a5000 2024-09-16T15:07:06.913 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb0ffff640 1 -- 172.21.15.133:0/830560452 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100a5e90 con 0x7feb180a5000 2024-09-16T15:07:06.914 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- 172.21.15.133:0/830560452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a5000 msgr2=0x7feb180a8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:06.914 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 --2- 172.21.15.133:0/830560452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a5000 0x7feb180a8f40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feb10009dc0 tx=0x7feb1001c8e0 comp rx=0 tx=0).stop 2024-09-16T15:07:06.914 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- 172.21.15.133:0/830560452 shutdown_connections 2024-09-16T15:07:06.914 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 --2- 172.21.15.133:0/830560452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a5000 0x7feb180a8f40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:06.914 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- 172.21.15.133:0/830560452 >> 172.21.15.133:0/830560452 conn(0x7feb1806f0e0 msgr2=0x7feb1806f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:06.914 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- 172.21.15.133:0/830560452 shutdown_connections 2024-09-16T15:07:06.914 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- 172.21.15.133:0/830560452 wait complete. 2024-09-16T15:07:06.915 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 Processor -- start 2024-09-16T15:07:06.915 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- start start 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb180a5000 0x7feb181439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb180a6df0 0x7feb18136b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a7de0 0x7feb18142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1813cda0 con 0x7feb180a6df0 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb18144090 con 0x7feb180a5000 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb18144350 con 0x7feb180a7de0 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1c84b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb180a6df0 0x7feb18136b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d84d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a7de0 0x7feb18142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d84d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a7de0 0x7feb18142560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:42846/0 (socket says 172.21.15.133:42846) 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d84d640 1 -- 172.21.15.133:0/1292425662 learned_addr learned my addr 172.21.15.133:0/1292425662 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:06.917 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1c84b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb180a6df0 0x7feb18136b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53288/0 (socket says 172.21.15.133:53288) 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d04c640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb180a5000 0x7feb181439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d84d640 1 -- 172.21.15.133:0/1292425662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb180a5000 msgr2=0x7feb181439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d84d640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb180a5000 0x7feb181439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d84d640 1 -- 172.21.15.133:0/1292425662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb180a6df0 msgr2=0x7feb18136b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d84d640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb180a6df0 0x7feb18136b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d84d640 1 -- 172.21.15.133:0/1292425662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb10009a70 con 0x7feb180a7de0 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1d84d640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a7de0 0x7feb18142560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feb1001cdc0 tx=0x7feb10019680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb0dffb640 1 -- 172.21.15.133:0/1292425662 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10004450 con 0x7feb180a7de0 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb0dffb640 1 -- 172.21.15.133:0/1292425662 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb100045b0 con 0x7feb180a7de0 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb0dffb640 1 -- 172.21.15.133:0/1292425662 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10009450 con 0x7feb180a7de0 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- 172.21.15.133:0/1292425662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb1813cfa0 con 0x7feb180a7de0 2024-09-16T15:07:06.918 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.910+0000 7feb1f2b0640 1 -- 172.21.15.133:0/1292425662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb1809ef40 con 0x7feb180a7de0 2024-09-16T15:07:06.919 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.914+0000 7feb1f2b0640 1 -- 172.21.15.133:0/1292425662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb180a8f40 con 0x7feb180a7de0 2024-09-16T15:07:06.919 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.914+0000 7feb0dffb640 1 -- 172.21.15.133:0/1292425662 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 4) v1 ==== 128821+0+0 (secure 0 0 0) 0x7feb100b4030 con 0x7feb180a7de0 2024-09-16T15:07:06.919 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.914+0000 7feb0dffb640 20 mgrc handle_mgr_map mgrmap(e 4) v1 2024-09-16T15:07:06.919 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.914+0000 7feb0dffb640 4 mgrc handle_mgr_map Got map version 4 2024-09-16T15:07:06.920 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.914+0000 7feb0dffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:06.920 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.914+0000 7feb0dffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:06.920 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.914+0000 7feb0dffb640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7feafc0977e0 0x7feafc099ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:06.920 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.914+0000 7feb0dffb640 1 -- 172.21.15.133:0/1292425662 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1623+0+0 (secure 0 0 0) 0x7feb1015a430 con 0x7feb180a7de0 2024-09-16T15:07:06.920 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.914+0000 7feb1d04c640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7feafc0977e0 0x7feafc099ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:06.920 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.914+0000 7feb1d04c640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7feafc0977e0 0x7feafc099ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb140011f0 tx=0x7feb14007d40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:06.923 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:06.918+0000 7feb0dffb640 1 -- 172.21.15.133:0/1292425662 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feb101287f0 con 0x7feb180a7de0 2024-09-16T15:07:07.082 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.078+0000 7feb1f2b0640 1 -- 172.21.15.133:0/1292425662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb18066480 con 0x7feb180a7de0 2024-09-16T15:07:07.083 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.078+0000 7feb0dffb640 1 -- 172.21.15.133:0/1292425662 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+4152 (secure 0 0 0) 0x7feb1012a720 con 0x7feb180a7de0 2024-09-16T15:07:07.084 INFO:teuthology.misc.health.smithi133.stdout: 2024-09-16T15:07:07.084 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":8,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:05.750356+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 10 mgrc shutdown 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 -- 172.21.15.133:0/1292425662 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7feafc0977e0 msgr2=0x7feafc099ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7feafc0977e0 0x7feafc099ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb140011f0 tx=0x7feb14007d40 comp rx=0 tx=0).stop 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 -- 172.21.15.133:0/1292425662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a7de0 msgr2=0x7feb18142560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a7de0 0x7feb18142560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feb1001cdc0 tx=0x7feb10019680 comp rx=0 tx=0).stop 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 -- 172.21.15.133:0/1292425662 shutdown_connections 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb180a7de0 0x7feb18142560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7feafc0977e0 0x7feafc099ca0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb180a6df0 0x7feb18136b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 --2- 172.21.15.133:0/1292425662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb180a5000 0x7feb181439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:07.087 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 -- 172.21.15.133:0/1292425662 >> 172.21.15.133:0/1292425662 conn(0x7feb1806f0e0 msgr2=0x7feb180b6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:07.088 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 -- 172.21.15.133:0/1292425662 shutdown_connections 2024-09-16T15:07:07.088 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:07.082+0000 7feb1f2b0640 1 -- 172.21.15.133:0/1292425662 wait complete. 2024-09-16T15:07:07.101 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-16T15:07:07.430 INFO:tasks.ceph.osd.1.smithi191.stderr:2024-09-16T15:07:07.428+0000 7f1048fc8640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T15:07:07.445 INFO:tasks.ceph.osd.3.smithi191.stderr:2024-09-16T15:07:07.444+0000 7fd410cc4640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T15:07:07.457 INFO:tasks.ceph.osd.1.smithi191.stderr:2024-09-16T15:07:07.456+0000 7f104160d640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:07:07.457 INFO:tasks.ceph.osd.2.smithi191.stderr:2024-09-16T15:07:07.456+0000 7fcbf4dff640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T15:07:07.469 INFO:tasks.ceph.osd.3.smithi191.stderr:2024-09-16T15:07:07.468+0000 7fd409309640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:07:07.481 INFO:tasks.ceph.osd.2.smithi191.stderr:2024-09-16T15:07:07.480+0000 7fcbed444640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:07:13.506 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:13.633 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d3812d640 1 Processor -- start 2024-09-16T15:07:13.633 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d3812d640 1 -- start start 2024-09-16T15:07:13.634 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d3812d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d300aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.634 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d3812d640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8d30135c10 0x7f8d300a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d3812d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d30136150 con 0x7f8d300a5fd0 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d3812d640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d30136380 con 0x7f8d30135c10 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d3812d640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30139d10 con 0x7f8d300a99f0 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d35ec9640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d300aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d366ca640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8d300a5fd0 0x7f8d300b9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37670/0 (socket says 172.21.15.133:37670) 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d35ec9640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d300aaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:38138/0 (socket says 172.21.15.133:38138) 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d366ca640 1 -- 172.21.15.133:0/689471646 learned_addr learned my addr 172.21.15.133:0/689471646 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d356c8640 1 --2- 172.21.15.133:0/689471646 >> v2:172.21.15.191:3300/0 conn(0x7f8d30135c10 0x7f8d300a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d34ec7640 1 -- 172.21.15.133:0/689471646 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692807886 0 0) 0x7f8d30136150 con 0x7f8d300a5fd0 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d35ec9640 1 -- 172.21.15.133:0/689471646 >> v2:172.21.15.191:3300/0 conn(0x7f8d30135c10 msgr2=0x7f8d300a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d35ec9640 1 --2- 172.21.15.133:0/689471646 >> v2:172.21.15.191:3300/0 conn(0x7f8d30135c10 0x7f8d300a84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d35ec9640 1 -- 172.21.15.133:0/689471646 >> v1:172.21.15.133:6789/0 conn(0x7f8d300a5fd0 legacy=0x7f8d300b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:13.635 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d35ec9640 1 -- 172.21.15.133:0/689471646 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300a9dd0 con 0x7f8d300a99f0 2024-09-16T15:07:13.637 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d35ec9640 1 --2- 172.21.15.133:0/689471646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d300aaee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0097c0 tx=0x7f8d2c01c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=393b918686b0a262 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:13.637 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.630+0000 7f8d34ec7640 1 -- 172.21.15.133:0/689471646 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c0ab600 con 0x7f8d300a99f0 2024-09-16T15:07:13.637 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d34ec7640 1 -- 172.21.15.133:0/689471646 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c0a7050 con 0x7f8d300a99f0 2024-09-16T15:07:13.637 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d34ec7640 1 -- 172.21.15.133:0/689471646 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c0ab980 con 0x7f8d300a99f0 2024-09-16T15:07:13.638 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- 172.21.15.133:0/689471646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 msgr2=0x7f8d300aaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:13.638 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 --2- 172.21.15.133:0/689471646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d300aaee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0097c0 tx=0x7f8d2c01c360 comp rx=0 tx=0).stop 2024-09-16T15:07:13.638 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- 172.21.15.133:0/689471646 shutdown_connections 2024-09-16T15:07:13.638 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 --2- 172.21.15.133:0/689471646 >> v2:172.21.15.191:3300/0 conn(0x7f8d30135c10 0x7f8d300a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.638 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 --2- 172.21.15.133:0/689471646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d300aaee0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.638 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- 172.21.15.133:0/689471646 >> 172.21.15.133:0/689471646 conn(0x7f8d3006af20 msgr2=0x7f8d3006bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:13.638 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- 172.21.15.133:0/689471646 shutdown_connections 2024-09-16T15:07:13.638 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- 172.21.15.133:0/689471646 wait complete. 2024-09-16T15:07:13.640 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 Processor -- start 2024-09-16T15:07:13.640 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- start start 2024-09-16T15:07:13.640 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d300a5fd0 0x7f8d30098ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.640 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d301df460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.640 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d30135c10 0x7f8d301df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3009f8f0 con 0x7f8d30135c10 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30135240 con 0x7f8d300a5fd0 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3009ffc0 con 0x7f8d300a99f0 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d356c8640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d301df460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d356c8640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d301df460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:38152/0 (socket says 172.21.15.133:38152) 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d366ca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d30135c10 0x7f8d301df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d356c8640 1 -- 172.21.15.133:0/4194795620 learned_addr learned my addr 172.21.15.133:0/4194795620 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d35ec9640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d300a5fd0 0x7f8d30098ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d366ca640 1 -- 172.21.15.133:0/4194795620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 msgr2=0x7f8d301df460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d366ca640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d301df460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d366ca640 1 -- 172.21.15.133:0/4194795620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d300a5fd0 msgr2=0x7f8d30098ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d366ca640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d300a5fd0 0x7f8d30098ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d366ca640 1 -- 172.21.15.133:0/4194795620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c009420 con 0x7f8d30135c10 2024-09-16T15:07:13.641 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d356c8640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d301df460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:13.642 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d366ca640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d30135c10 0x7f8d301df9a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8d28002240 tx=0x7f8d2800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:13.642 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d267fc640 1 -- 172.21.15.133:0/4194795620 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2800ce30 con 0x7f8d30135c10 2024-09-16T15:07:13.642 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d267fc640 1 -- 172.21.15.133:0/4194795620 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2800a640 con 0x7f8d30135c10 2024-09-16T15:07:13.642 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- 172.21.15.133:0/4194795620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d30099390 con 0x7f8d30135c10 2024-09-16T15:07:13.642 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d267fc640 1 -- 172.21.15.133:0/4194795620 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2800a9f0 con 0x7f8d30135c10 2024-09-16T15:07:13.642 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.634+0000 7f8d3812d640 1 -- 172.21.15.133:0/4194795620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d300a87e0 con 0x7f8d30135c10 2024-09-16T15:07:13.642 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.638+0000 7f8d3812d640 1 -- 172.21.15.133:0/4194795620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d300672a0 con 0x7f8d30135c10 2024-09-16T15:07:13.643 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.638+0000 7f8d267fc640 1 -- 172.21.15.133:0/4194795620 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f8d2800ab50 con 0x7f8d30135c10 2024-09-16T15:07:13.643 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.638+0000 7f8d267fc640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:13.643 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.638+0000 7f8d267fc640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:13.644 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.638+0000 7f8d267fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:13.644 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.638+0000 7f8d267fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:13.644 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.638+0000 7f8d267fc640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8d14097840 0x7f8d14099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.644 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.638+0000 7f8d267fc640 1 -- 172.21.15.133:0/4194795620 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3129+0+0 (secure 0 0 0) 0x7f8d280bf180 con 0x7f8d30135c10 2024-09-16T15:07:13.644 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.638+0000 7f8d35ec9640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8d14097840 0x7f8d14099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.644 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.638+0000 7f8d35ec9640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8d14097840 0x7f8d14099d00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0ad000 tx=0x7f8d2c0a6000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:13.647 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.642+0000 7f8d267fc640 1 -- 172.21.15.133:0/4194795620 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8d2808cea0 con 0x7f8d30135c10 2024-09-16T15:07:13.804 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.798+0000 7f8d3812d640 1 -- 172.21.15.133:0/4194795620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d300aaee0 con 0x7f8d30135c10 2024-09-16T15:07:13.806 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d267fc640 1 -- 172.21.15.133:0/4194795620 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+7929 (secure 0 0 0) 0x7f8d28018070 con 0x7f8d30135c10 2024-09-16T15:07:13.806 INFO:teuthology.misc.health.smithi133.stdout: 2024-09-16T15:07:13.806 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":13,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:11.590870+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:13.808 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 10 mgrc shutdown 2024-09-16T15:07:13.808 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 -- 172.21.15.133:0/4194795620 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8d14097840 msgr2=0x7f8d14099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:13.808 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8d14097840 0x7f8d14099d00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0ad000 tx=0x7f8d2c0a6000 comp rx=0 tx=0).stop 2024-09-16T15:07:13.809 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 -- 172.21.15.133:0/4194795620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d30135c10 msgr2=0x7f8d301df9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:13.809 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d30135c10 0x7f8d301df9a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8d28002240 tx=0x7f8d2800cbe0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.809 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 -- 172.21.15.133:0/4194795620 shutdown_connections 2024-09-16T15:07:13.809 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8d14097840 0x7f8d14099d00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.809 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d30135c10 0x7f8d301df9a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.809 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d300a99f0 0x7f8d301df460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.809 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 --2- 172.21.15.133:0/4194795620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d300a5fd0 0x7f8d30098ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.809 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 -- 172.21.15.133:0/4194795620 >> 172.21.15.133:0/4194795620 conn(0x7f8d3006af20 msgr2=0x7f8d300a75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:13.809 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 -- 172.21.15.133:0/4194795620 shutdown_connections 2024-09-16T15:07:13.809 INFO:teuthology.misc.health.smithi133.stderr:2024-09-16T15:07:13.802+0000 7f8d3812d640 1 -- 172.21.15.133:0/4194795620 wait complete. 2024-09-16T15:07:13.825 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-16T15:07:13.825 INFO:tasks.ceph:Creating RBD pool 2024-09-16T15:07:13.825 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T15:07:13.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.934+0000 7f9e0aed7640 1 Processor -- start 2024-09-16T15:07:13.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.934+0000 7f9e0aed7640 1 -- start start 2024-09-16T15:07:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e0aed7640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e0aed7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e0406f400 0x7f9e0406bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e0aed7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e04066340 con 0x7f9e0406b920 2024-09-16T15:07:13.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e0aed7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0406c260 con 0x7f9e0406f400 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e0aed7640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0406b180 con 0x7f9e04065f60 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e08c73640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e09474640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9e0406f400 0x7f9e0406bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e03fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9e0406b920 0x7f9e0406ecf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37674/0 (socket says 172.21.15.133:37674) 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e03fff640 1 -- 172.21.15.133:0/1262079515 learned_addr learned my addr 172.21.15.133:0/1262079515 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e08c73640 1 --2- 172.21.15.133:0/1262079515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040a3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:38154/0 (socket says 172.21.15.133:38154) 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e09474640 1 -- 172.21.15.133:0/1262079515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 msgr2=0x7f9e040a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e09474640 1 --2- 172.21.15.133:0/1262079515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040a3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e09474640 1 -- 172.21.15.133:0/1262079515 >> v1:172.21.15.133:6789/0 conn(0x7f9e0406b920 legacy=0x7f9e0406ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e09474640 1 -- 172.21.15.133:0/1262079515 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0406b440 con 0x7f9e0406f400 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e037fe640 1 -- 172.21.15.133:0/1262079515 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341453077 0 0) 0x7f9e04066340 con 0x7f9e0406b920 2024-09-16T15:07:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e08c73640 1 --2- 172.21.15.133:0/1262079515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:13.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e09474640 1 --2- 172.21.15.133:0/1262079515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e0406f400 0x7f9e0406bd20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009d90 tx=0x7f9dfc01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d213db25e2e532f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:13.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e037fe640 1 -- 172.21.15.133:0/1262079515 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0a5890 con 0x7f9e0406f400 2024-09-16T15:07:13.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e037fe640 1 -- 172.21.15.133:0/1262079515 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc0a59f0 con 0x7f9e0406f400 2024-09-16T15:07:13.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.938+0000 7f9e037fe640 1 -- 172.21.15.133:0/1262079515 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0a5d80 con 0x7f9e0406f400 2024-09-16T15:07:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- 172.21.15.133:0/1262079515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e0406f400 msgr2=0x7f9e0406bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 --2- 172.21.15.133:0/1262079515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e0406f400 0x7f9e0406bd20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009d90 tx=0x7f9dfc01c8c0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- 172.21.15.133:0/1262079515 shutdown_connections 2024-09-16T15:07:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 --2- 172.21.15.133:0/1262079515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e0406f400 0x7f9e0406bd20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 --2- 172.21.15.133:0/1262079515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- 172.21.15.133:0/1262079515 >> 172.21.15.133:0/1262079515 conn(0x7f9e0406a030 msgr2=0x7f9e0406a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- 172.21.15.133:0/1262079515 shutdown_connections 2024-09-16T15:07:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- 172.21.15.133:0/1262079515 wait complete. 2024-09-16T15:07:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 Processor -- start 2024-09-16T15:07:13.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- start start 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e0406b920 0x7f9e0412bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e0406f400 0x7f9e04149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0412c5e0 con 0x7f9e0406b920 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0414a290 con 0x7f9e0406f400 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0414a780 con 0x7f9e04065f60 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e08c73640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e08c73640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:38162/0 (socket says 172.21.15.133:38162) 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e08c73640 1 -- 172.21.15.133:0/2792824614 learned_addr learned my addr 172.21.15.133:0/2792824614 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e03fff640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e0406b920 0x7f9e0412bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e09474640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e0406f400 0x7f9e04149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e08c73640 1 -- 172.21.15.133:0/2792824614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e0406f400 msgr2=0x7f9e04149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e08c73640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e0406f400 0x7f9e04149d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e08c73640 1 -- 172.21.15.133:0/2792824614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e0406b920 msgr2=0x7f9e0412bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e08c73640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e0406b920 0x7f9e0412bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e08c73640 1 -- 172.21.15.133:0/2792824614 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8009420 con 0x7f9e04065f60 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e08c73640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040ae870 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9df800ebe0 tx=0x7f9df800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e017fa640 1 -- 172.21.15.133:0/2792824614 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df800eee0 con 0x7f9e04065f60 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e017fa640 1 -- 172.21.15.133:0/2792824614 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df800cc30 con 0x7f9e04065f60 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e017fa640 1 -- 172.21.15.133:0/2792824614 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8013470 con 0x7f9e04065f60 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- 172.21.15.133:0/2792824614 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc009a70 con 0x7f9e04065f60 2024-09-16T15:07:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- 172.21.15.133:0/2792824614 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e040b4b80 con 0x7f9e04065f60 2024-09-16T15:07:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.942+0000 7f9e0aed7640 1 -- 172.21.15.133:0/2792824614 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e040672a0 con 0x7f9e04065f60 2024-09-16T15:07:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.946+0000 7f9e017fa640 1 -- 172.21.15.133:0/2792824614 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f9df80135d0 con 0x7f9e04065f60 2024-09-16T15:07:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.946+0000 7f9e017fa640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.946+0000 7f9e017fa640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.946+0000 7f9e017fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.946+0000 7f9e017fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.946+0000 7f9e017fa640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9df0097840 0x7f9df0099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.946+0000 7f9e017fa640 1 -- 172.21.15.133:0/2792824614 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3129+0+0 (secure 0 0 0) 0x7f9df80c02c0 con 0x7f9e04065f60 2024-09-16T15:07:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.946+0000 7f9e03fff640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9df0097840 0x7f9df0099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.946+0000 7f9e03fff640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9df0097840 0x7f9df0099d00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9df4007fd0 tx=0x7f9df4007d60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:13.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:13.950+0000 7f9e017fa640 1 -- 172.21.15.133:0/2792824614 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9df808dea0 con 0x7f9e04065f60 2024-09-16T15:07:14.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.102+0000 7f9e0aed7640 1 -- 172.21.15.133:0/2792824614 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f9e04149710 con 0x7f9e04065f60 2024-09-16T15:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.618+0000 7f9e017fa640 1 -- 172.21.15.133:0/2792824614 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f9df8021020 con 0x7f9e04065f60 2024-09-16T15:07:14.624 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'rbd' created 2024-09-16T15:07:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 10 mgrc shutdown 2024-09-16T15:07:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 -- 172.21.15.133:0/2792824614 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9df0097840 msgr2=0x7f9df0099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9df0097840 0x7f9df0099d00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9df4007fd0 tx=0x7f9df4007d60 comp rx=0 tx=0).stop 2024-09-16T15:07:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 -- 172.21.15.133:0/2792824614 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 msgr2=0x7f9e040ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040ae870 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9df800ebe0 tx=0x7f9df800c520 comp rx=0 tx=0).stop 2024-09-16T15:07:14.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 -- 172.21.15.133:0/2792824614 shutdown_connections 2024-09-16T15:07:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e0406f400 0x7f9e04149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9df0097840 0x7f9df0099d00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9e0406b920 0x7f9e0412bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 --2- 172.21.15.133:0/2792824614 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9e04065f60 0x7f9e040ae870 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 -- 172.21.15.133:0/2792824614 >> 172.21.15.133:0/2792824614 conn(0x7f9e0406a030 msgr2=0x7f9e040b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 -- 172.21.15.133:0/2792824614 shutdown_connections 2024-09-16T15:07:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.622+0000 7f9e0aed7640 1 -- 172.21.15.133:0/2792824614 wait complete. 2024-09-16T15:07:14.648 DEBUG:teuthology.orchestra.run.smithi133:> rbd --cluster ceph pool init rbd 2024-09-16T15:07:14.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f9894ae4e00 1 Processor -- start 2024-09-16T15:07:14.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f9894ae4e00 1 -- start start 2024-09-16T15:07:14.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f9894ae4e00 1 --2- >> v2:172.21.15.191:3300/0 conn(0x563f1ca43250 0x563f1ca51930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:14.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f9894ae4e00 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x563f1c8cdab0 con 0x563f1ca41a80 2024-09-16T15:07:14.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f9894ae4e00 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x563f1c8a69c0 con 0x563f1cad1610 2024-09-16T15:07:14.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f9894ae4e00 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x563f1c910370 con 0x563f1ca43250 2024-09-16T15:07:14.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f9892dfe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x563f1ca41a80 0x563f1cad0f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:37676/0 (socket says 172.21.15.133:37676) 2024-09-16T15:07:14.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f9892dfe640 1 -- 172.21.15.133:0/3592877863 learned_addr learned my addr 172.21.15.133:0/3592877863 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:14.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f98935ff640 1 --2- 172.21.15.133:0/3592877863 >> v2:172.21.15.191:3300/0 conn(0x563f1ca43250 0x563f1ca51930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:14.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f98925fd640 1 -- 172.21.15.133:0/3592877863 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153798811 0 0) 0x563f1c8cdab0 con 0x563f1ca41a80 2024-09-16T15:07:14.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f98925fd640 1 -- 172.21.15.133:0/3592877863 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f987c001e20 con 0x563f1ca41a80 2024-09-16T15:07:14.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f98935ff640 1 -- 172.21.15.133:0/3592877863 >> v1:172.21.15.191:6789/0 conn(0x563f1cad1610 legacy=0x563f1cae6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:14.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f98935ff640 1 -- 172.21.15.133:0/3592877863 >> v1:172.21.15.133:6789/0 conn(0x563f1ca41a80 legacy=0x563f1cad0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:14.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.682+0000 7f98935ff640 1 -- 172.21.15.133:0/3592877863 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f1c8b3910 con 0x563f1ca43250 2024-09-16T15:07:14.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f98935ff640 1 --2- 172.21.15.133:0/3592877863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f1ca43250 0x563f1ca51930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f988400a500 tx=0x7f988401b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8f98d4b0b73b83 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:14.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f98925fd640 1 -- 172.21.15.133:0/3592877863 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840a6660 con 0x563f1ca43250 2024-09-16T15:07:14.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f98925fd640 1 -- 172.21.15.133:0/3592877863 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98840a67c0 con 0x563f1ca43250 2024-09-16T15:07:14.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f98925fd640 1 -- 172.21.15.133:0/3592877863 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840a6b50 con 0x563f1ca43250 2024-09-16T15:07:14.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 -- 172.21.15.133:0/3592877863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f1ca43250 msgr2=0x563f1ca51930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:14.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 --2- 172.21.15.133:0/3592877863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f1ca43250 0x563f1ca51930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f988400a500 tx=0x7f988401b840 comp rx=0 tx=0).stop 2024-09-16T15:07:14.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 -- 172.21.15.133:0/3592877863 shutdown_connections 2024-09-16T15:07:14.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 --2- 172.21.15.133:0/3592877863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f1ca43250 0x563f1ca51930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:14.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 -- 172.21.15.133:0/3592877863 >> 172.21.15.133:0/3592877863 conn(0x563f1ca22220 msgr2=0x563f1ca1aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:14.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 -- 172.21.15.133:0/3592877863 shutdown_connections 2024-09-16T15:07:14.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 -- 172.21.15.133:0/3592877863 wait complete. 2024-09-16T15:07:14.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 Processor -- start 2024-09-16T15:07:14.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 -- start start 2024-09-16T15:07:14.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f1ca41a80 0x563f1cacce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:14.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f1ca43250 0x563f1cacaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:14.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563f1cad1610 0x563f1cacb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:14.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f1c90d940 con 0x563f1ca43250 2024-09-16T15:07:14.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f1c909230 con 0x563f1ca41a80 2024-09-16T15:07:14.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.686+0000 7f9894ae4e00 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f1c8a49b0 con 0x563f1cad1610 2024-09-16T15:07:14.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9892dfe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f1ca43250 0x563f1cacaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:14.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9894861640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563f1cad1610 0x563f1cacb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9892dfe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f1ca43250 0x563f1cacaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39950/0 (socket says 172.21.15.133:39950) 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9892dfe640 1 -- 172.21.15.133:0/503960466 learned_addr learned my addr 172.21.15.133:0/503960466 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98935ff640 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f1ca41a80 0x563f1cacce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9894861640 1 -- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f1ca41a80 msgr2=0x563f1cacce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9894861640 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563f1ca41a80 0x563f1cacce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9894861640 1 -- 172.21.15.133:0/503960466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f1ca43250 msgr2=0x563f1cacaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9894861640 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f1ca43250 0x563f1cacaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9894861640 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9884005d30 con 0x563f1cad1610 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9892dfe640 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f1ca43250 0x563f1cacaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9894861640 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563f1cad1610 0x563f1cacb1e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f988c006f90 tx=0x7f988c00b4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c006230 con 0x563f1cad1610 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c005450 con 0x563f1cad1610 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563f1c8a6d50 con 0x563f1cad1610 2024-09-16T15:07:14.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c005800 con 0x563f1cad1610 2024-09-16T15:07:14.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f1c8a67b0 con 0x563f1cad1610 2024-09-16T15:07:14.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f988c005960 con 0x563f1cad1610 2024-09-16T15:07:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98837fe640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98837fe640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98837fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98837fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98837fe640 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9878096130 0x7f98780985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3471+0+0 (secure 0 0 0) 0x7f988c0bdbd0 con 0x563f1cad1610 2024-09-16T15:07:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f98935ff640 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9878096130 0x7f98780985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.690+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x563f1cae16a0 con 0x563f1cad1610 2024-09-16T15:07:14.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:14.694+0000 7f98935ff640 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9878096130 0x7f98780985f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98840af000 tx=0x7f98840b2040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.638+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 141+0+0 (secure 0 0 0) 0x7f988c0170a0 con 0x563f1cad1610 2024-09-16T15:07:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.638+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x563f1c8ce650 con 0x563f1cad1610 2024-09-16T15:07:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.638+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f988c006b60 con 0x563f1cad1610 2024-09-16T15:07:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.638+0000 7f9891dfc640 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=15}) v3 -- 0x7f9870002af0 con 0x563f1cad1610 2024-09-16T15:07:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.638+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 8 ==== osd_map(15..15 src has 1..15) v4 ==== 650+0+0 (secure 0 0 0) 0x7f988c08fab0 con 0x563f1cad1610 2024-09-16T15:07:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.638+0000 7f9894ae4e00 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] conn(0x563f1cacbc30 0x563f1caf8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:15.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.638+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) v8 -- 0x563f1caf90a0 con 0x563f1cacbc30 2024-09-16T15:07:15.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.638+0000 7f9892dfe640 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] conn(0x563f1cacbc30 0x563f1caf8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:15.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.638+0000 7f9892dfe640 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] conn(0x563f1cacbc30 0x563f1caf8b60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:15.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.654+0000 7f9892dfe640 1 -- 172.21.15.133:0/503960466 <== osd.3 v2:172.21.15.191:6800/787707562 1 ==== osd_op_reply(1 rbd_trash [create] v15'1 uv1 ondisk = 0) v8 ==== 153+0+0 (crc 0 0 0) 0x7f988800ca20 con 0x563f1cacbc30 2024-09-16T15:07:15.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.658+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x563f1cb487f0 con 0x563f1cacbc30 2024-09-16T15:07:15.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.662+0000 7f9892dfe640 1 -- 172.21.15.133:0/503960466 <== osd.3 v2:172.21.15.191:6800/787707562 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f988800ca20 con 0x563f1cacbc30 2024-09-16T15:07:15.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.662+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e15) v8 -- 0x563f1cb47900 con 0x563f1cacbc30 2024-09-16T15:07:15.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.662+0000 7f9892dfe640 1 -- 172.21.15.133:0/503960466 <== osd.3 v2:172.21.15.191:6800/787707562 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7f988800ca20 con 0x563f1cacbc30 2024-09-16T15:07:15.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:15.662+0000 7f98915fb640 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) v4 -- 0x7f9874001f30 con 0x563f1cad1610 2024-09-16T15:07:16.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:16.638+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 9 ==== pool_op_reply(tid 4 (0) Success v16) v1 ==== 55+0+0 (secure 0 0 0) 0x7f988c08b660 con 0x563f1cad1610 2024-09-16T15:07:16.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:16.638+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=16}) v3 -- 0x7f987809bb70 con 0x563f1cad1610 2024-09-16T15:07:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:16.638+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 10 ==== osd_map(16..16 src has 1..16) v4 ==== 626+0+0 (secure 0 0 0) 0x7f988c08f3b0 con 0x563f1cad1610 2024-09-16T15:07:16.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:16.642+0000 7f9891dfc640 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) v8 -- 0x7f9870005610 con 0x563f1cacbc30 2024-09-16T15:07:16.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:16.646+0000 7f9892dfe640 1 -- 172.21.15.133:0/503960466 <== osd.3 v2:172.21.15.191:6800/787707562 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v16'1 uv1 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7f988800ca20 con 0x563f1cacbc30 2024-09-16T15:07:16.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:16.646+0000 7f98915fb640 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v16) v4 -- 0x7f98740033b0 con 0x563f1cad1610 2024-09-16T15:07:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.658+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 11 ==== pool_op_reply(tid 6 (0) Success v17) v1 ==== 43+0+0 (secure 0 0 0) 0x7f988c08d590 con 0x563f1cad1610 2024-09-16T15:07:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.658+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=17}) v3 -- 0x7f987809bd60 con 0x563f1cad1610 2024-09-16T15:07:17.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.658+0000 7f98837fe640 1 -- 172.21.15.133:0/503960466 <== mon.2 v2:172.21.15.133:3301/0 12 ==== osd_map(17..17 src has 1..17) v4 ==== 654+0+0 (secure 0 0 0) 0x7f988c0931e0 con 0x563f1cad1610 2024-09-16T15:07:17.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.658+0000 7f9891dfc640 1 -- 172.21.15.133:0/503960466 --> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) v8 -- 0x7f9870005a00 con 0x563f1cacbc30 2024-09-16T15:07:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9892dfe640 1 -- 172.21.15.133:0/503960466 <== osd.3 v2:172.21.15.191:6800/787707562 5 ==== osd_op_reply(7 rbd_info [write 0~19] v17'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7f988800ca20 con 0x563f1cacbc30 2024-09-16T15:07:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] conn(0x563f1cacbc30 msgr2=0x563f1caf8b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] conn(0x563f1cacbc30 0x563f1caf8b60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 10 mgrc shutdown 2024-09-16T15:07:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9878096130 msgr2=0x7f98780985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9878096130 0x7f98780985f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98840af000 tx=0x7f98840b2040 comp rx=0 tx=0).stop 2024-09-16T15:07:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563f1cad1610 msgr2=0x563f1cacb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 1 --2- 172.21.15.133:0/503960466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563f1cad1610 0x563f1cacb1e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f988c006f90 tx=0x7f988c00b4b0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894861640 1 -- 172.21.15.133:0/503960466 reap_dead start 2024-09-16T15:07:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 shutdown_connections 2024-09-16T15:07:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 >> 172.21.15.133:0/503960466 conn(0x563f1ca22220 msgr2=0x563f1cacdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 shutdown_connections 2024-09-16T15:07:17.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.670+0000 7f9894ae4e00 1 -- 172.21.15.133:0/503960466 wait complete. 2024-09-16T15:07:17.689 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T15:07:17.690 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:07:17.690 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:07:17.735 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:07:17.735 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:07:17.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:07:17.737 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:07:17.737 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:07:17.738 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:07:17.740 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:07:17.740 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-16T15:07:17.741 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:17.817 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-09-16T15:07:17.818 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-09-16T15:07:17.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.854+0000 7f2ec9696640 1 Processor -- start 2024-09-16T15:07:17.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.854+0000 7f2ec9696640 1 -- start start 2024-09-16T15:07:17.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec9696640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2ec414cc40 0x7f2ec414f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec9696640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec40a8b70 con 0x7f2ec40a8790 2024-09-16T15:07:17.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec9696640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec40a6a40 con 0x7f2ec4139a10 2024-09-16T15:07:17.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec9696640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec40ab280 con 0x7f2ec414cc40 2024-09-16T15:07:17.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec2ffd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2ec40a8790 0x7f2ec40aa980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52440/0 (socket says 172.21.15.133:52440) 2024-09-16T15:07:17.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec2ffd640 1 -- 172.21.15.133:0/44792030 learned_addr learned my addr 172.21.15.133:0/44792030 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:17.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec37fe640 1 --2- 172.21.15.133:0/44792030 >> v2:172.21.15.191:3300/0 conn(0x7f2ec414cc40 0x7f2ec414f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856551233 0 0) 0x7f2ec40a8b70 con 0x7f2ec40a8790 2024-09-16T15:07:17.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eac003530 con 0x7f2ec40a8790 2024-09-16T15:07:17.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971172637 0 0) 0x7f2eac003530 con 0x7f2ec40a8790 2024-09-16T15:07:17.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 >> v2:172.21.15.191:3300/0 conn(0x7f2ec414cc40 msgr2=0x7f2ec414f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:17.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 --2- 172.21.15.133:0/44792030 >> v2:172.21.15.191:3300/0 conn(0x7f2ec414cc40 0x7f2ec414f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 >> v1:172.21.15.191:6789/0 conn(0x7f2ec4139a10 legacy=0x7f2ec4147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:17.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec41468f0 con 0x7f2ec40a8790 2024-09-16T15:07:17.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f2eb4001a00 con 0x7f2ec40a8790 2024-09-16T15:07:17.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 >> v1:172.21.15.133:6789/0 conn(0x7f2ec40a8790 legacy=0x7f2ec40aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:17.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2eac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2eac0047d0 0x7f2eac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2eac0052b0 0x7f2eac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40ab280 con 0x7f2eac003980 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac00c180 con 0x7f2eac0047d0 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac00c340 con 0x7f2eac0052b0 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec2ffd640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2eac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec37fe640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2eac0052b0 0x7f2eac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec27fc640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2eac0047d0 0x7f2eac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec2ffd640 1 -- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2eac0052b0 msgr2=0x7f2eac00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec2ffd640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2eac0052b0 0x7f2eac00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec2ffd640 1 -- 172.21.15.133:0/44792030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2eac0047d0 msgr2=0x7f2eac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:17.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec2ffd640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2eac0047d0 0x7f2eac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec2ffd640 1 -- 172.21.15.133:0/44792030 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac003530 con 0x7f2eac003980 2024-09-16T15:07:17.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec37fe640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2eac0052b0 0x7f2eac00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:17.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec37fe640 1 -- 172.21.15.133:0/44792030 reap_dead start 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec2ffd640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2eac003da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400a7f0 tx=0x7f2eb401d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea999a6a6a6ee510 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40072d0 con 0x7f2eac003980 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb40a8070 con 0x7f2eac003980 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.858+0000 7f2ec1ffb640 1 -- 172.21.15.133:0/44792030 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb40076f0 con 0x7f2eac003980 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 msgr2=0x7f2eac003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2eac003da0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400a7f0 tx=0x7f2eb401d1e0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- 172.21.15.133:0/44792030 shutdown_connections 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 --2- 172.21.15.133:0/44792030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2eac003da0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- 172.21.15.133:0/44792030 >> 172.21.15.133:0/44792030 conn(0x7f2ec406a260 msgr2=0x7f2ec406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- 172.21.15.133:0/44792030 shutdown_connections 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- 172.21.15.133:0/44792030 wait complete. 2024-09-16T15:07:17.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 Processor -- start 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- start start 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2ec4135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ec41d1d30 0x7f2ec41df610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ec41dfb50 0x7f2ec41dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec41d26a0 con 0x7f2eac003980 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec41dd9a0 con 0x7f2ec41dfb50 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec41ddc60 con 0x7f2ec41d1d30 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2ec4135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec27fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ec41d1d30 0x7f2ec41df610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2ec4135560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51386/0 (socket says 172.21.15.133:51386) 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec2ffd640 1 -- 172.21.15.133:0/3134191608 learned_addr learned my addr 172.21.15.133:0/3134191608 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec37fe640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ec41dfb50 0x7f2ec41dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec27fc640 1 -- 172.21.15.133:0/3134191608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ec41dfb50 msgr2=0x7f2ec41dffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec27fc640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ec41dfb50 0x7f2ec41dffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec27fc640 1 -- 172.21.15.133:0/3134191608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 msgr2=0x7f2ec4135560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec27fc640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2ec4135560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec27fc640 1 -- 172.21.15.133:0/3134191608 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec40a5e70 con 0x7f2ec41d1d30 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec2ffd640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2ec4135560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec27fc640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ec41d1d30 0x7f2ec41df610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800e440 tx=0x7f2eb800e900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2eab7fe640 1 -- 172.21.15.133:0/3134191608 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80052e0 con 0x7f2ec41d1d30 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2eab7fe640 1 -- 172.21.15.133:0/3134191608 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb8005440 con 0x7f2ec41d1d30 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- 172.21.15.133:0/3134191608 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec4134b00 con 0x7f2ec41d1d30 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2eab7fe640 1 -- 172.21.15.133:0/3134191608 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8003840 con 0x7f2ec41d1d30 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.862+0000 7f2ec9696640 1 -- 172.21.15.133:0/3134191608 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec41369c0 con 0x7f2ec41d1d30 2024-09-16T15:07:17.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.866+0000 7f2ec9696640 1 -- 172.21.15.133:0/3134191608 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec40a80e0 con 0x7f2ec41d1d30 2024-09-16T15:07:17.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.866+0000 7f2eab7fe640 1 -- 172.21.15.133:0/3134191608 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f2eb80039a0 con 0x7f2ec41d1d30 2024-09-16T15:07:17.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.866+0000 7f2eab7fe640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:17.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.866+0000 7f2eab7fe640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:17.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.866+0000 7f2eab7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:17.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.866+0000 7f2eab7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:17.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.866+0000 7f2eab7fe640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2eb0097870 0x7f2eb0099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:17.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.866+0000 7f2eab7fe640 1 -- 172.21.15.133:0/3134191608 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f2eb80bef70 con 0x7f2ec41d1d30 2024-09-16T15:07:17.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.866+0000 7f2ec2ffd640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2eb0097870 0x7f2eb0099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:17.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.866+0000 7f2ec2ffd640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2eb0097870 0x7f2eb0099d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4001cc0 tx=0x7f2eb4009050 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:17.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:17.870+0000 7f2eab7fe640 1 -- 172.21.15.133:0/3134191608 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2eb80f0db0 con 0x7f2ec41d1d30 2024-09-16T15:07:18.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.030+0000 7f2ec9696640 1 -- 172.21.15.133:0/3134191608 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ec40ab630 con 0x7f2ec41d1d30 2024-09-16T15:07:18.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.030+0000 7f2eab7fe640 1 -- 172.21.15.133:0/3134191608 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+9772 (secure 0 0 0) 0x7f2eb8090580 con 0x7f2ec41d1d30 2024-09-16T15:07:18.035 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:18.035 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:17.648059+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 10 mgrc shutdown 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 -- 172.21.15.133:0/3134191608 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2eb0097870 msgr2=0x7f2eb0099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2eb0097870 0x7f2eb0099d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4001cc0 tx=0x7f2eb4009050 comp rx=0 tx=0).stop 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 -- 172.21.15.133:0/3134191608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ec41d1d30 msgr2=0x7f2ec41df610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ec41d1d30 0x7f2ec41df610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800e440 tx=0x7f2eb800e900 comp rx=0 tx=0).stop 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 -- 172.21.15.133:0/3134191608 shutdown_connections 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ec41dfb50 0x7f2ec41dffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ec41d1d30 0x7f2ec41df610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2eb0097870 0x7f2eb0099d30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 --2- 172.21.15.133:0/3134191608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2eac003980 0x7f2ec4135560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 -- 172.21.15.133:0/3134191608 >> 172.21.15.133:0/3134191608 conn(0x7f2ec406a260 msgr2=0x7f2ec40a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:18.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 -- 172.21.15.133:0/3134191608 shutdown_connections 2024-09-16T15:07:18.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.034+0000 7f2ec9696640 1 -- 172.21.15.133:0/3134191608 wait complete. 2024-09-16T15:07:18.049 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-16T15:07:18.055 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:07:08.758832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T15:07:14.143631+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:07:18.055 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:07:18.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd6aa0640 1 Processor -- start 2024-09-16T15:07:18.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd6aa0640 1 -- start start 2024-09-16T15:07:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd6aa0640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a2cf0 0x7f0fd00a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd6aa0640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0fd009e490 0x7f0fd006afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd6aa0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0fd00a1010 0x7f0fd0144a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd6aa0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd006a8f0 con 0x7f0fd00a1010 2024-09-16T15:07:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd6aa0640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd00a5bd0 con 0x7f0fd009e490 2024-09-16T15:07:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd6aa0640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd00a5910 con 0x7f0fd00a2cf0 2024-09-16T15:07:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd483c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a2cf0 0x7f0fd00a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd483c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a2cf0 0x7f0fd00a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:33904/0 (socket says 172.21.15.133:33904) 2024-09-16T15:07:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd483c640 1 -- 172.21.15.133:0/107021286 learned_addr learned my addr 172.21.15.133:0/107021286 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:18.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd503d640 1 --2- 172.21.15.133:0/107021286 >> v2:172.21.15.133:3300/0 conn(0x7f0fd00a1010 0x7f0fd0144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fc7fff640 1 --2- 172.21.15.133:0/107021286 >> v2:172.21.15.191:3300/0 conn(0x7f0fd009e490 0x7f0fd006afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd483c640 1 -- 172.21.15.133:0/107021286 >> v2:172.21.15.191:3300/0 conn(0x7f0fd009e490 msgr2=0x7f0fd006afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:18.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd483c640 1 --2- 172.21.15.133:0/107021286 >> v2:172.21.15.191:3300/0 conn(0x7f0fd009e490 0x7f0fd006afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd483c640 1 -- 172.21.15.133:0/107021286 >> v2:172.21.15.133:3300/0 conn(0x7f0fd00a1010 msgr2=0x7f0fd0144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:18.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd483c640 1 --2- 172.21.15.133:0/107021286 >> v2:172.21.15.133:3300/0 conn(0x7f0fd00a1010 0x7f0fd0144a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.174+0000 7f0fd483c640 1 -- 172.21.15.133:0/107021286 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00a4cf0 con 0x7f0fd00a2cf0 2024-09-16T15:07:18.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd483c640 1 --2- 172.21.15.133:0/107021286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a2cf0 0x7f0fd00a53d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0fd006a530 tx=0x7f0fcc01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ced9b4a9329035 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:18.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fc77fe640 1 -- 172.21.15.133:0/107021286 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0ac600 con 0x7f0fd00a2cf0 2024-09-16T15:07:18.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fc77fe640 1 -- 172.21.15.133:0/107021286 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fcc0a8050 con 0x7f0fd00a2cf0 2024-09-16T15:07:18.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fc77fe640 1 -- 172.21.15.133:0/107021286 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0ac980 con 0x7f0fd00a2cf0 2024-09-16T15:07:18.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/107021286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a2cf0 msgr2=0x7f0fd00a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 --2- 172.21.15.133:0/107021286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a2cf0 0x7f0fd00a53d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0fd006a530 tx=0x7f0fcc01c3e0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/107021286 shutdown_connections 2024-09-16T15:07:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 --2- 172.21.15.133:0/107021286 >> v2:172.21.15.133:3300/0 conn(0x7f0fd00a1010 0x7f0fd0144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 --2- 172.21.15.133:0/107021286 >> v2:172.21.15.191:3300/0 conn(0x7f0fd009e490 0x7f0fd006afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 --2- 172.21.15.133:0/107021286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a2cf0 0x7f0fd00a53d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/107021286 >> 172.21.15.133:0/107021286 conn(0x7f0fd00a3fd0 msgr2=0x7f0fd00a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/107021286 shutdown_connections 2024-09-16T15:07:18.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/107021286 wait complete. 2024-09-16T15:07:18.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 Processor -- start 2024-09-16T15:07:18.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 -- start start 2024-09-16T15:07:18.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd009e490 0x7f0fd01dfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a1010 0x7f0fd0148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fd00a2cf0 0x7f0fd0149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd01dd010 con 0x7f0fd009e490 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd01df870 con 0x7f0fd00a2cf0 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd6aa0640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd01dc270 con 0x7f0fd00a1010 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd483c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd009e490 0x7f0fd01dfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fc7fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a1010 0x7f0fd0148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd483c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd009e490 0x7f0fd01dfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51400/0 (socket says 172.21.15.133:51400) 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd483c640 1 -- 172.21.15.133:0/1002633523 learned_addr learned my addr 172.21.15.133:0/1002633523 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd503d640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fd00a2cf0 0x7f0fd0149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd483c640 1 -- 172.21.15.133:0/1002633523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a1010 msgr2=0x7f0fd0148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd483c640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a1010 0x7f0fd0148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd483c640 1 -- 172.21.15.133:0/1002633523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fd00a2cf0 msgr2=0x7f0fd0149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd483c640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fd00a2cf0 0x7f0fd0149270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.178+0000 7f0fd483c640 1 -- 172.21.15.133:0/1002633523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc0094d0 con 0x7f0fd009e490 2024-09-16T15:07:18.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fd483c640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd009e490 0x7f0fd01dfcf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc01cc80 tx=0x7f0fcc019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:18.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc57fa640 1 -- 172.21.15.133:0/1002633523 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0b5460 con 0x7f0fd009e490 2024-09-16T15:07:18.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc57fa640 1 -- 172.21.15.133:0/1002633523 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fcc0b55c0 con 0x7f0fd009e490 2024-09-16T15:07:18.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/1002633523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd01dc510 con 0x7f0fd009e490 2024-09-16T15:07:18.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc57fa640 1 -- 172.21.15.133:0/1002633523 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0b5970 con 0x7f0fd009e490 2024-09-16T15:07:18.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/1002633523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd01e0230 con 0x7f0fd009e490 2024-09-16T15:07:18.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/1002633523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd0144a60 con 0x7f0fd009e490 2024-09-16T15:07:18.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc57fa640 1 -- 172.21.15.133:0/1002633523 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f0fcc0b3070 con 0x7f0fd009e490 2024-09-16T15:07:18.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc57fa640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:18.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc57fa640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:18.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc57fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:18.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc57fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:18.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc57fa640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0fbc097840 0x7f0fbc099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc57fa640 1 -- 172.21.15.133:0/1002633523 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f0fcc15a550 con 0x7f0fd009e490 2024-09-16T15:07:18.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.182+0000 7f0fc7fff640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0fbc097840 0x7f0fbc099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.186+0000 7f0fc7fff640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0fbc097840 0x7f0fbc099d00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0148760 tx=0x7f0fc0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:18.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.186+0000 7f0fc57fa640 1 -- 172.21.15.133:0/1002633523 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0fcc128020 con 0x7f0fd009e490 2024-09-16T15:07:18.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/1002633523 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0fd00b4a80 con 0x7f0fd009e490 2024-09-16T15:07:18.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fc57fa640 1 -- 172.21.15.133:0/1002633523 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0fcc0b10a0 con 0x7f0fd009e490 2024-09-16T15:07:18.350 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-09-16T15:07:18.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 10 mgrc shutdown 2024-09-16T15:07:18.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/1002633523 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0fbc097840 msgr2=0x7f0fbc099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0fbc097840 0x7f0fbc099d00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0148760 tx=0x7f0fc0008040 comp rx=0 tx=0).stop 2024-09-16T15:07:18.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/1002633523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd009e490 msgr2=0x7f0fd01dfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd009e490 0x7f0fd01dfcf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc01cc80 tx=0x7f0fcc019180 comp rx=0 tx=0).stop 2024-09-16T15:07:18.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/1002633523 shutdown_connections 2024-09-16T15:07:18.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fd00a2cf0 0x7f0fd0149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fd00a1010 0x7f0fd0148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0fbc097840 0x7f0fbc099d00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 --2- 172.21.15.133:0/1002633523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd009e490 0x7f0fd01dfcf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/1002633523 >> 172.21.15.133:0/1002633523 conn(0x7f0fd00a3fd0 msgr2=0x7f0fd0072d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:18.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/1002633523 shutdown_connections 2024-09-16T15:07:18.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.346+0000 7f0fd6aa0640 1 -- 172.21.15.133:0/1002633523 wait complete. 2024-09-16T15:07:18.369 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-16T15:07:18.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.486+0000 7fe079e11640 1 Processor -- start 2024-09-16T15:07:18.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.486+0000 7fe079e11640 1 -- start start 2024-09-16T15:07:18.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe079e11640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0740a6210 0x7fe07409edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe079e11640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe0740a2fe0 0x7fe074140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe079e11640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe074141110 0x7fe074143970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe079e11640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07409f310 con 0x7fe0740a6210 2024-09-16T15:07:18.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe079e11640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0740a3590 con 0x7fe0740a2fe0 2024-09-16T15:07:18.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe079e11640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0740a3850 con 0x7fe074141110 2024-09-16T15:07:18.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe0737fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0740a6210 0x7fe07409edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe073fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe074141110 0x7fe074143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe0737fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0740a6210 0x7fe07409edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51410/0 (socket says 172.21.15.133:51410) 2024-09-16T15:07:18.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe073fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe074141110 0x7fe074143970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:33916/0 (socket says 172.21.15.133:33916) 2024-09-16T15:07:18.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe0737fe640 1 -- 172.21.15.133:0/579482239 learned_addr learned my addr 172.21.15.133:0/579482239 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:18.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe06bfff640 1 --2- 172.21.15.133:0/579482239 >> v2:172.21.15.191:3300/0 conn(0x7fe0740a2fe0 0x7fe074140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe073fff640 1 -- 172.21.15.133:0/579482239 >> v2:172.21.15.191:3300/0 conn(0x7fe0740a2fe0 msgr2=0x7fe074140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:18.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe073fff640 1 --2- 172.21.15.133:0/579482239 >> v2:172.21.15.191:3300/0 conn(0x7fe0740a2fe0 0x7fe074140bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe073fff640 1 -- 172.21.15.133:0/579482239 >> v2:172.21.15.133:3300/0 conn(0x7fe0740a6210 msgr2=0x7fe07409edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:18.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe073fff640 1 --2- 172.21.15.133:0/579482239 >> v2:172.21.15.133:3300/0 conn(0x7fe0740a6210 0x7fe07409edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe073fff640 1 -- 172.21.15.133:0/579482239 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0741415c0 con 0x7fe074141110 2024-09-16T15:07:18.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe0737fe640 1 --2- 172.21.15.133:0/579482239 >> v2:172.21.15.133:3300/0 conn(0x7fe0740a6210 0x7fe07409edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:18.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe073fff640 1 --2- 172.21.15.133:0/579482239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe074141110 0x7fe074143970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe0740a3fa0 tx=0x7fe06c01e740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dadbdd07c5cfd0d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:18.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe072ffd640 1 -- 172.21.15.133:0/579482239 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0a77f0 con 0x7fe074141110 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe072ffd640 1 -- 172.21.15.133:0/579482239 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c0a7950 con 0x7fe074141110 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.490+0000 7fe072ffd640 1 -- 172.21.15.133:0/579482239 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0a7ce0 con 0x7fe074141110 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- 172.21.15.133:0/579482239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe074141110 msgr2=0x7fe074143970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 --2- 172.21.15.133:0/579482239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe074141110 0x7fe074143970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe0740a3fa0 tx=0x7fe06c01e740 comp rx=0 tx=0).stop 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- 172.21.15.133:0/579482239 shutdown_connections 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 --2- 172.21.15.133:0/579482239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe074141110 0x7fe074143970 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 --2- 172.21.15.133:0/579482239 >> v2:172.21.15.191:3300/0 conn(0x7fe0740a2fe0 0x7fe074140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 --2- 172.21.15.133:0/579482239 >> v2:172.21.15.133:3300/0 conn(0x7fe0740a6210 0x7fe07409edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- 172.21.15.133:0/579482239 >> 172.21.15.133:0/579482239 conn(0x7fe0740a3fd0 msgr2=0x7fe0740a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- 172.21.15.133:0/579482239 shutdown_connections 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- 172.21.15.133:0/579482239 wait complete. 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 Processor -- start 2024-09-16T15:07:18.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- start start 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0740a2fe0 0x7fe0740a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0740a6210 0x7fe074149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe074141110 0x7fe0741db5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0741dc030 con 0x7fe0740a6210 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0741dc230 con 0x7fe074141110 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe074149c10 con 0x7fe0740a2fe0 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe0737fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0740a2fe0 0x7fe0740a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe06bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0740a6210 0x7fe074149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe0737fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0740a2fe0 0x7fe0740a8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:33932/0 (socket says 172.21.15.133:33932) 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe0737fe640 1 -- 172.21.15.133:0/751277895 learned_addr learned my addr 172.21.15.133:0/751277895 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:18.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe073fff640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe074141110 0x7fe0741db5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe0737fe640 1 -- 172.21.15.133:0/751277895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe074141110 msgr2=0x7fe0741db5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe0737fe640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe074141110 0x7fe0741db5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe0737fe640 1 -- 172.21.15.133:0/751277895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0740a6210 msgr2=0x7fe074149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe0737fe640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0740a6210 0x7fe074149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe0737fe640 1 -- 172.21.15.133:0/751277895 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c0094d0 con 0x7fe0740a2fe0 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe06bfff640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0740a6210 0x7fe074149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe0737fe640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0740a2fe0 0x7fe0740a8ca0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00ea00 tx=0x7fe05c00eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe070ff9640 1 -- 172.21.15.133:0/751277895 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00ca60 con 0x7fe0740a2fe0 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe070ff9640 1 -- 172.21.15.133:0/751277895 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c00cbc0 con 0x7fe0740a2fe0 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- 172.21.15.133:0/751277895 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06c009a70 con 0x7fe0740a2fe0 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe070ff9640 1 -- 172.21.15.133:0/751277895 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c013470 con 0x7fe0740a2fe0 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- 172.21.15.133:0/751277895 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0740a9530 con 0x7fe0740a2fe0 2024-09-16T15:07:18.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.494+0000 7fe079e11640 1 -- 172.21.15.133:0/751277895 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07409edd0 con 0x7fe0740a2fe0 2024-09-16T15:07:18.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.498+0000 7fe070ff9640 1 -- 172.21.15.133:0/751277895 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fe05c0135d0 con 0x7fe0740a2fe0 2024-09-16T15:07:18.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.498+0000 7fe070ff9640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:18.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.498+0000 7fe070ff9640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:18.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.498+0000 7fe070ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:18.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.498+0000 7fe070ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:18.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.498+0000 7fe070ff9640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe054097840 0x7fe054099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.498+0000 7fe070ff9640 1 -- 172.21.15.133:0/751277895 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3538+0+0 (secure 0 0 0) 0x7fe05c0c03c0 con 0x7fe0740a2fe0 2024-09-16T15:07:18.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.498+0000 7fe06bfff640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe054097840 0x7fe054099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.498+0000 7fe06bfff640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe054097840 0x7fe054099d00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0741dc9b0 tx=0x7fe064008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:18.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.502+0000 7fe070ff9640 1 -- 172.21.15.133:0/751277895 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe05c08de00 con 0x7fe0740a2fe0 2024-09-16T15:07:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.662+0000 7fe079e11640 1 -- 172.21.15.133:0/751277895 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe07409e510 con 0x7fe0740a2fe0 2024-09-16T15:07:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.662+0000 7fe070ff9640 1 -- 172.21.15.133:0/751277895 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe05c021020 con 0x7fe0740a2fe0 2024-09-16T15:07:18.667 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 8 2024-09-16T15:07:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 10 mgrc shutdown 2024-09-16T15:07:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 -- 172.21.15.133:0/751277895 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe054097840 msgr2=0x7fe054099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe054097840 0x7fe054099d00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0741dc9b0 tx=0x7fe064008040 comp rx=0 tx=0).stop 2024-09-16T15:07:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 -- 172.21.15.133:0/751277895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0740a2fe0 msgr2=0x7fe0740a8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0740a2fe0 0x7fe0740a8ca0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00ea00 tx=0x7fe05c00eec0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 -- 172.21.15.133:0/751277895 shutdown_connections 2024-09-16T15:07:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe074141110 0x7fe0741db5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe054097840 0x7fe054099d00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0740a6210 0x7fe074149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 --2- 172.21.15.133:0/751277895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0740a2fe0 0x7fe0740a8ca0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 -- 172.21.15.133:0/751277895 >> 172.21.15.133:0/751277895 conn(0x7fe0740a3fd0 msgr2=0x7fe0740a1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:18.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 -- 172.21.15.133:0/751277895 shutdown_connections 2024-09-16T15:07:18.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.666+0000 7fe079e11640 1 -- 172.21.15.133:0/751277895 wait complete. 2024-09-16T15:07:18.686 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:07:18.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f96a5806640 1 Processor -- start 2024-09-16T15:07:18.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f96a5806640 1 -- start start 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f96a5806640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96a00a3980 0x7f96a006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f96a5806640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f96a006ae30 0x7f96a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f96a5806640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a00a57a0 con 0x7f96a00a66c0 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f96a5806640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a01438e0 con 0x7f96a00a3980 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f96a5806640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a00a3d60 con 0x7f96a006ae30 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f969effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96a00a3980 0x7f96a006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f969effd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96a00a3980 0x7f96a006a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51440/0 (socket says 172.21.15.133:51440) 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f969effd640 1 -- 172.21.15.133:0/107250176 learned_addr learned my addr 172.21.15.133:0/107250176 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f969e7fc640 1 --2- 172.21.15.133:0/107250176 >> v2:172.21.15.191:3300/0 conn(0x7f96a006ae30 0x7f96a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f969effd640 1 -- 172.21.15.133:0/107250176 >> v2:172.21.15.191:3300/0 conn(0x7f96a006ae30 msgr2=0x7f96a00a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:18.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f969effd640 1 --2- 172.21.15.133:0/107250176 >> v2:172.21.15.191:3300/0 conn(0x7f96a006ae30 0x7f96a00a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f969effd640 1 -- 172.21.15.133:0/107250176 >> v1:172.21.15.191:6789/0 conn(0x7f96a00a66c0 legacy=0x7f96a0143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:18.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.794+0000 7f969effd640 1 -- 172.21.15.133:0/107250176 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a01435f0 con 0x7f96a00a3980 2024-09-16T15:07:18.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.798+0000 7f969effd640 1 --2- 172.21.15.133:0/107250176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a3980 0x7f96a006a8f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96a006bf20 tx=0x7f968801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f3dc21e0266672 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:18.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.798+0000 7f969dffb640 1 -- 172.21.15.133:0/107250176 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96880aa5f0 con 0x7f96a00a3980 2024-09-16T15:07:18.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.798+0000 7f969dffb640 1 -- 172.21.15.133:0/107250176 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96880aa750 con 0x7f96a00a3980 2024-09-16T15:07:18.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.798+0000 7f969dffb640 1 -- 172.21.15.133:0/107250176 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96880aaae0 con 0x7f96a00a3980 2024-09-16T15:07:18.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- 172.21.15.133:0/107250176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a3980 msgr2=0x7f96a006a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 --2- 172.21.15.133:0/107250176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a3980 0x7f96a006a8f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f96a006bf20 tx=0x7f968801c470 comp rx=0 tx=0).stop 2024-09-16T15:07:18.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- 172.21.15.133:0/107250176 shutdown_connections 2024-09-16T15:07:18.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 --2- 172.21.15.133:0/107250176 >> v2:172.21.15.191:3300/0 conn(0x7f96a006ae30 0x7f96a00a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 --2- 172.21.15.133:0/107250176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a3980 0x7f96a006a8f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- 172.21.15.133:0/107250176 >> 172.21.15.133:0/107250176 conn(0x7f96a00a3fd0 msgr2=0x7f96a00a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:18.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- 172.21.15.133:0/107250176 shutdown_connections 2024-09-16T15:07:18.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- 172.21.15.133:0/107250176 wait complete. 2024-09-16T15:07:18.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 Processor -- start 2024-09-16T15:07:18.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- start start 2024-09-16T15:07:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a006ae30 0x7f96a01dbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a66c0 0x7f96a0147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a01490c0 0x7f96a009aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a0148c50 con 0x7f96a00a66c0 2024-09-16T15:07:18.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a01dc180 con 0x7f96a006ae30 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a01dc400 con 0x7f96a01490c0 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a66c0 0x7f96a0147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a66c0 0x7f96a0147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51454/0 (socket says 172.21.15.133:51454) 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969e7fc640 1 -- 172.21.15.133:0/2750375462 learned_addr learned my addr 172.21.15.133:0/2750375462 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969f7fe640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a01490c0 0x7f96a009aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969effd640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a006ae30 0x7f96a01dbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969e7fc640 1 -- 172.21.15.133:0/2750375462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a01490c0 msgr2=0x7f96a009aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969e7fc640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a01490c0 0x7f96a009aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969e7fc640 1 -- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a006ae30 msgr2=0x7f96a01dbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969e7fc640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a006ae30 0x7f96a01dbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969e7fc640 1 -- 172.21.15.133:0/2750375462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880094d0 con 0x7f96a00a66c0 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969effd640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a006ae30 0x7f96a01dbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:18.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f969e7fc640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a66c0 0x7f96a0147c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f969000d800 tx=0x7f969000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:18.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f9697fff640 1 -- 172.21.15.133:0/2750375462 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969000b6c0 con 0x7f96a00a66c0 2024-09-16T15:07:18.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f9697fff640 1 -- 172.21.15.133:0/2750375462 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969000b860 con 0x7f96a00a66c0 2024-09-16T15:07:18.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- 172.21.15.133:0/2750375462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a01dc6c0 con 0x7f96a00a66c0 2024-09-16T15:07:18.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f9697fff640 1 -- 172.21.15.133:0/2750375462 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969000bc10 con 0x7f96a00a66c0 2024-09-16T15:07:18.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.802+0000 7f96a5806640 1 -- 172.21.15.133:0/2750375462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a009b7b0 con 0x7f96a00a66c0 2024-09-16T15:07:18.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.806+0000 7f96a5806640 1 -- 172.21.15.133:0/2750375462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a006a8f0 con 0x7f96a00a66c0 2024-09-16T15:07:18.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.806+0000 7f9697fff640 1 -- 172.21.15.133:0/2750375462 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f969001e460 con 0x7f96a00a66c0 2024-09-16T15:07:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.806+0000 7f9697fff640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.806+0000 7f9697fff640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.806+0000 7f9697fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.806+0000 7f9697fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:18.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.806+0000 7f9697fff640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9680097840 0x7f9680099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:18.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.806+0000 7f9697fff640 1 -- 172.21.15.133:0/2750375462 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f96900c0970 con 0x7f96a00a66c0 2024-09-16T15:07:18.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.806+0000 7f969effd640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9680097840 0x7f9680099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:18.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.806+0000 7f969effd640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9680097840 0x7f9680099d00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f96880ae000 tx=0x7f9688008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:18.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:18.810+0000 7f9697fff640 1 -- 172.21.15.133:0/2750375462 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f969008e440 con 0x7f96a00a66c0 2024-09-16T15:07:19.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.006+0000 7f96a5806640 1 -- 172.21.15.133:0/2750375462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96a006b800 con 0x7f96a00a66c0 2024-09-16T15:07:19.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.006+0000 7f9697fff640 1 -- 172.21.15.133:0/2750375462 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2774 (secure 0 0 0) 0x7f9690091a60 con 0x7f96a00a66c0 2024-09-16T15:07:19.013 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:19.013 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6811/3104644688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3104644688},{"type":"v1","addr":"172.21.15.133:6811","nonce":3104644688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6809/147821810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":147821810},{"type":"v1","addr":"172.21.15.133:6809","nonce":147821810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/2751563681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2751563681},{"type":"v1","addr":"172.21.15.191:6827","nonce":2751563681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-16T15:07:19.013 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 3 2024-09-16T15:07:19.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 10 mgrc shutdown 2024-09-16T15:07:19.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 -- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9680097840 msgr2=0x7f9680099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9680097840 0x7f9680099d00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f96880ae000 tx=0x7f9688008040 comp rx=0 tx=0).stop 2024-09-16T15:07:19.015 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 -- 172.21.15.133:0/2750375462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a66c0 msgr2=0x7f96a0147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a66c0 0x7f96a0147c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f969000d800 tx=0x7f969000dcc0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 -- 172.21.15.133:0/2750375462 shutdown_connections 2024-09-16T15:07:19.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a01490c0 0x7f96a009aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9680097840 0x7f9680099d00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a00a66c0 0x7f96a0147c50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 --2- 172.21.15.133:0/2750375462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a006ae30 0x7f96a01dbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 -- 172.21.15.133:0/2750375462 >> 172.21.15.133:0/2750375462 conn(0x7f96a00a3fd0 msgr2=0x7f96a01450b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:19.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 -- 172.21.15.133:0/2750375462 shutdown_connections 2024-09-16T15:07:19.016 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.010+0000 7f96a5806640 1 -- 172.21.15.133:0/2750375462 wait complete. 2024-09-16T15:07:19.030 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.130+0000 7f8a4eef2640 1 Processor -- start 2024-09-16T15:07:19.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.130+0000 7f8a4eef2640 1 -- start start 2024-09-16T15:07:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.130+0000 7f8a4eef2640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a480a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.130+0000 7f8a4eef2640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8a4809e490 0x7f8a4806afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.130+0000 7f8a4eef2640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4806a8f0 con 0x7f8a480a1010 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.130+0000 7f8a4eef2640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a480a5bd0 con 0x7f8a4809e490 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.130+0000 7f8a4eef2640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a480a5910 con 0x7f8a480a2cf0 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.134+0000 7f8a4cc8e640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a480a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.134+0000 7f8a4cc8e640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a480a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:33948/0 (socket says 172.21.15.133:33948) 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.134+0000 7f8a4d48f640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8a480a1010 0x7f8a48144a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52444/0 (socket says 172.21.15.133:52444) 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.134+0000 7f8a4cc8e640 1 -- 172.21.15.133:0/2083164785 learned_addr learned my addr 172.21.15.133:0/2083164785 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.134+0000 7f8a47fff640 1 --2- 172.21.15.133:0/2083164785 >> v2:172.21.15.191:3300/0 conn(0x7f8a4809e490 0x7f8a4806afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.134+0000 7f8a477fe640 1 -- 172.21.15.133:0/2083164785 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192638073 0 0) 0x7f8a4806a8f0 con 0x7f8a480a1010 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.134+0000 7f8a4cc8e640 1 -- 172.21.15.133:0/2083164785 >> v2:172.21.15.191:3300/0 conn(0x7f8a4809e490 msgr2=0x7f8a4806afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.134+0000 7f8a4cc8e640 1 --2- 172.21.15.133:0/2083164785 >> v2:172.21.15.191:3300/0 conn(0x7f8a4809e490 0x7f8a4806afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.134+0000 7f8a4cc8e640 1 -- 172.21.15.133:0/2083164785 >> v1:172.21.15.133:6789/0 conn(0x7f8a480a1010 legacy=0x7f8a48144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.134+0000 7f8a4cc8e640 1 -- 172.21.15.133:0/2083164785 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a480a4cf0 con 0x7f8a480a2cf0 2024-09-16T15:07:19.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4cc8e640 1 --2- 172.21.15.133:0/2083164785 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a480a53d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009870 tx=0x7f8a3801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a3ec48674b6616c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a477fe640 1 -- 172.21.15.133:0/2083164785 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a380ab620 con 0x7f8a480a2cf0 2024-09-16T15:07:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a477fe640 1 -- 172.21.15.133:0/2083164785 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a380ab780 con 0x7f8a480a2cf0 2024-09-16T15:07:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a477fe640 1 -- 172.21.15.133:0/2083164785 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a380abb10 con 0x7f8a480a2cf0 2024-09-16T15:07:19.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 -- 172.21.15.133:0/2083164785 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 msgr2=0x7f8a480a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 --2- 172.21.15.133:0/2083164785 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a480a53d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009870 tx=0x7f8a3801c420 comp rx=0 tx=0).stop 2024-09-16T15:07:19.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 -- 172.21.15.133:0/2083164785 shutdown_connections 2024-09-16T15:07:19.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 --2- 172.21.15.133:0/2083164785 >> v2:172.21.15.191:3300/0 conn(0x7f8a4809e490 0x7f8a4806afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 --2- 172.21.15.133:0/2083164785 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a480a53d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 -- 172.21.15.133:0/2083164785 >> 172.21.15.133:0/2083164785 conn(0x7f8a480a3fd0 msgr2=0x7f8a480a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:19.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 -- 172.21.15.133:0/2083164785 shutdown_connections 2024-09-16T15:07:19.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 -- 172.21.15.133:0/2083164785 wait complete. 2024-09-16T15:07:19.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 Processor -- start 2024-09-16T15:07:19.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 -- start start 2024-09-16T15:07:19.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4809e490 0x7f8a481469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a480a1010 0x7f8a4809ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a4809b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48148b80 con 0x7f8a4809e490 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4809d320 con 0x7f8a480a1010 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4eef2640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4809d5c0 con 0x7f8a480a2cf0 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4cc8e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4809e490 0x7f8a481469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4d48f640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a4809b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4cc8e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4809e490 0x7f8a481469a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51460/0 (socket says 172.21.15.133:51460) 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4cc8e640 1 -- 172.21.15.133:0/4245140544 learned_addr learned my addr 172.21.15.133:0/4245140544 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a47fff640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a480a1010 0x7f8a4809ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4cc8e640 1 -- 172.21.15.133:0/4245140544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 msgr2=0x7f8a4809b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4cc8e640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a4809b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4cc8e640 1 -- 172.21.15.133:0/4245140544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a480a1010 msgr2=0x7f8a4809ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4cc8e640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a480a1010 0x7f8a4809ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4cc8e640 1 -- 172.21.15.133:0/4245140544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a380094d0 con 0x7f8a4809e490 2024-09-16T15:07:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4d48f640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a4809b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.138+0000 7f8a4cc8e640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4809e490 0x7f8a481469a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a380ad040 tx=0x7f8a380191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a457fa640 1 -- 172.21.15.133:0/4245140544 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a380bf410 con 0x7f8a4809e490 2024-09-16T15:07:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a457fa640 1 -- 172.21.15.133:0/4245140544 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a380bf5b0 con 0x7f8a4809e490 2024-09-16T15:07:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a4eef2640 1 -- 172.21.15.133:0/4245140544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4809d880 con 0x7f8a4809e490 2024-09-16T15:07:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a457fa640 1 -- 172.21.15.133:0/4245140544 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a380bf960 con 0x7f8a4809e490 2024-09-16T15:07:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a4eef2640 1 -- 172.21.15.133:0/4245140544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a48147580 con 0x7f8a4809e490 2024-09-16T15:07:19.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a4eef2640 1 -- 172.21.15.133:0/4245140544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4809c560 con 0x7f8a4809e490 2024-09-16T15:07:19.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a457fa640 1 -- 172.21.15.133:0/4245140544 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f8a380b5400 con 0x7f8a4809e490 2024-09-16T15:07:19.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a457fa640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:19.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a457fa640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:19.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a457fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:19.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a457fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:19.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a457fa640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8a34097870 0x7f8a34099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a457fa640 1 -- 172.21.15.133:0/4245140544 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f8a3815a3c0 con 0x7f8a4809e490 2024-09-16T15:07:19.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a47fff640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8a34097870 0x7f8a34099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.142+0000 7f8a47fff640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8a34097870 0x7f8a34099d30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8a480a4740 tx=0x7f8a3c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:19.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.146+0000 7f8a457fa640 1 -- 172.21.15.133:0/4245140544 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8a38127e90 con 0x7f8a4809e490 2024-09-16T15:07:19.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.294+0000 7f8a4eef2640 1 -- 172.21.15.133:0/4245140544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a4809c560 con 0x7f8a4809e490 2024-09-16T15:07:19.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.294+0000 7f8a457fa640 1 -- 172.21.15.133:0/4245140544 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+9772 (secure 0 0 0) 0x7f8a4809c560 con 0x7f8a4809e490 2024-09-16T15:07:19.299 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:19.300 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:17.648059+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:19.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 10 mgrc shutdown 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 -- 172.21.15.133:0/4245140544 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8a34097870 msgr2=0x7f8a34099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8a34097870 0x7f8a34099d30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8a480a4740 tx=0x7f8a3c008040 comp rx=0 tx=0).stop 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 -- 172.21.15.133:0/4245140544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4809e490 msgr2=0x7f8a481469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4809e490 0x7f8a481469a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8a380ad040 tx=0x7f8a380191c0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 -- 172.21.15.133:0/4245140544 shutdown_connections 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a480a2cf0 0x7f8a4809b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a480a1010 0x7f8a4809ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8a34097870 0x7f8a34099d30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 --2- 172.21.15.133:0/4245140544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4809e490 0x7f8a481469a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 -- 172.21.15.133:0/4245140544 >> 172.21.15.133:0/4245140544 conn(0x7f8a480a3fd0 msgr2=0x7f8a480b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 -- 172.21.15.133:0/4245140544 shutdown_connections 2024-09-16T15:07:19.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.298+0000 7f8a4eef2640 1 -- 172.21.15.133:0/4245140544 wait complete. 2024-09-16T15:07:19.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:07:08.758832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T15:07:14.143631+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:07:19.317 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:07:19.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.434+0000 7f269d9bd640 1 Processor -- start 2024-09-16T15:07:19.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.434+0000 7f269d9bd640 1 -- start start 2024-09-16T15:07:19.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f269d9bd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980a8790 0x7f26980aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f269d9bd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f269814bc30 0x7f269814e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f269d9bd640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26980a8b70 con 0x7f2698139a10 2024-09-16T15:07:19.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f269d9bd640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26980a6a40 con 0x7f269814bc30 2024-09-16T15:07:19.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f269d9bd640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26980ab280 con 0x7f26980a8790 2024-09-16T15:07:19.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f26977fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f269814bc30 0x7f269814e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f2696ffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980a8790 0x7f26980aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f26977fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f269814bc30 0x7f269814e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51464/0 (socket says 172.21.15.133:51464) 2024-09-16T15:07:19.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f26977fe640 1 -- 172.21.15.133:0/4242034551 learned_addr learned my addr 172.21.15.133:0/4242034551 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:19.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f26977fe640 1 -- 172.21.15.133:0/4242034551 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980a8790 msgr2=0x7f26980aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:19.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f26977fe640 1 --2- 172.21.15.133:0/4242034551 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980a8790 0x7f26980aa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f26977fe640 1 -- 172.21.15.133:0/4242034551 >> v1:172.21.15.191:6789/0 conn(0x7f2698139a10 legacy=0x7f2698147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f26977fe640 1 -- 172.21.15.133:0/4242034551 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26981468f0 con 0x7f269814bc30 2024-09-16T15:07:19.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f26977fe640 1 --2- 172.21.15.133:0/4242034551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f269814bc30 0x7f269814e020 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2690009e80 tx=0x7f269001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a960777b53256ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:19.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f2695ffb640 1 -- 172.21.15.133:0/4242034551 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26900a5780 con 0x7f269814bc30 2024-09-16T15:07:19.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f2695ffb640 1 -- 172.21.15.133:0/4242034551 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26900a58e0 con 0x7f269814bc30 2024-09-16T15:07:19.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.438+0000 7f2695ffb640 1 -- 172.21.15.133:0/4242034551 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26900a5c70 con 0x7f269814bc30 2024-09-16T15:07:19.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- 172.21.15.133:0/4242034551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f269814bc30 msgr2=0x7f269814e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 --2- 172.21.15.133:0/4242034551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f269814bc30 0x7f269814e020 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2690009e80 tx=0x7f269001c8a0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- 172.21.15.133:0/4242034551 shutdown_connections 2024-09-16T15:07:19.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 --2- 172.21.15.133:0/4242034551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f269814bc30 0x7f269814e020 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 --2- 172.21.15.133:0/4242034551 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f26980a8790 0x7f26980aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- 172.21.15.133:0/4242034551 >> 172.21.15.133:0/4242034551 conn(0x7f269806a260 msgr2=0x7f269806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:19.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- 172.21.15.133:0/4242034551 shutdown_connections 2024-09-16T15:07:19.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- 172.21.15.133:0/4242034551 wait complete. 2024-09-16T15:07:19.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 Processor -- start 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- start start 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980a8790 0x7f2698131ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2698139a10 0x7f26980b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f269814bc30 0x7f2698148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2698132990 con 0x7f26980a8790 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26980a54a0 con 0x7f269814bc30 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2698132430 con 0x7f2698139a10 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f2696ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980a8790 0x7f2698131ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f2696ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980a8790 0x7f2698131ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51466/0 (socket says 172.21.15.133:51466) 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f2696ffd640 1 -- 172.21.15.133:0/3015071650 learned_addr learned my addr 172.21.15.133:0/3015071650 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f26967fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2698139a10 0x7f26980b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f2696ffd640 1 -- 172.21.15.133:0/3015071650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2698139a10 msgr2=0x7f26980b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f26977fe640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f269814bc30 0x7f2698148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f2696ffd640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2698139a10 0x7f26980b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f2696ffd640 1 -- 172.21.15.133:0/3015071650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f269814bc30 msgr2=0x7f2698148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f2696ffd640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f269814bc30 0x7f2698148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f2696ffd640 1 -- 172.21.15.133:0/3015071650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2690009b30 con 0x7f26980a8790 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f26967fc640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2698139a10 0x7f26980b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f2696ffd640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980a8790 0x7f2698131ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f268800e9e0 tx=0x7f268800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f267ffff640 1 -- 172.21.15.133:0/3015071650 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26880096b0 con 0x7f26980a8790 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f267ffff640 1 -- 172.21.15.133:0/3015071650 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268800c9d0 con 0x7f26980a8790 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- 172.21.15.133:0/3015071650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26981326d0 con 0x7f26980a8790 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f267ffff640 1 -- 172.21.15.133:0/3015071650 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268800cd60 con 0x7f26980a8790 2024-09-16T15:07:19.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- 172.21.15.133:0/3015071650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2698149d10 con 0x7f26980a8790 2024-09-16T15:07:19.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.442+0000 7f269d9bd640 1 -- 172.21.15.133:0/3015071650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26980aa980 con 0x7f26980a8790 2024-09-16T15:07:19.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.446+0000 7f267ffff640 1 -- 172.21.15.133:0/3015071650 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f2688013470 con 0x7f26980a8790 2024-09-16T15:07:19.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.446+0000 7f267ffff640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:19.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.446+0000 7f267ffff640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:19.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.446+0000 7f267ffff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:19.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.446+0000 7f267ffff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:19.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.446+0000 7f267ffff640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2684097940 0x7f2684099e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.446+0000 7f267ffff640 1 -- 172.21.15.133:0/3015071650 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f26880c0350 con 0x7f26980a8790 2024-09-16T15:07:19.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.446+0000 7f26967fc640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2684097940 0x7f2684099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.446+0000 7f26967fc640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2684097940 0x7f2684099e00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f26980a2ab0 tx=0x7f268c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:19.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.450+0000 7f267ffff640 1 -- 172.21.15.133:0/3015071650 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f268808de10 con 0x7f26980a8790 2024-09-16T15:07:19.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.606+0000 7f269d9bd640 1 -- 172.21.15.133:0/3015071650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26980ab630 con 0x7f26980a8790 2024-09-16T15:07:19.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.606+0000 7f267ffff640 1 -- 172.21.15.133:0/3015071650 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2688021020 con 0x7f26980a8790 2024-09-16T15:07:19.612 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-09-16T15:07:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 10 mgrc shutdown 2024-09-16T15:07:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 -- 172.21.15.133:0/3015071650 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2684097940 msgr2=0x7f2684099e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2684097940 0x7f2684099e00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f26980a2ab0 tx=0x7f268c009040 comp rx=0 tx=0).stop 2024-09-16T15:07:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 -- 172.21.15.133:0/3015071650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980a8790 msgr2=0x7f2698131ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980a8790 0x7f2698131ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f268800e9e0 tx=0x7f268800eea0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 -- 172.21.15.133:0/3015071650 shutdown_connections 2024-09-16T15:07:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f269814bc30 0x7f2698148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2698139a10 0x7f26980b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2684097940 0x7f2684099e00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 --2- 172.21.15.133:0/3015071650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26980a8790 0x7f2698131ef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 -- 172.21.15.133:0/3015071650 >> 172.21.15.133:0/3015071650 conn(0x7f269806a260 msgr2=0x7f26980a6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:19.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 -- 172.21.15.133:0/3015071650 shutdown_connections 2024-09-16T15:07:19.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.610+0000 7f269d9bd640 1 -- 172.21.15.133:0/3015071650 wait complete. 2024-09-16T15:07:19.631 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-16T15:07:19.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.746+0000 7fe1bf805640 1 Processor -- start 2024-09-16T15:07:19.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.746+0000 7fe1bf805640 1 -- start start 2024-09-16T15:07:19.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bf805640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b806afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bf805640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1b80a1010 0x7fe1b814bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bf805640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b806a8f0 con 0x7fe1b80a2cf0 2024-09-16T15:07:19.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bf805640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b80a5bd0 con 0x7fe1b80a1010 2024-09-16T15:07:19.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bf805640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80a4cf0 con 0x7fe1b809e490 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bd5a1640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe1b80a2cf0 0x7fe1b80a53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52456/0 (socket says 172.21.15.133:52456) 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bd5a1640 1 -- 172.21.15.133:0/1250719396 learned_addr learned my addr 172.21.15.133:0/1250719396 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bdda2640 1 --2- 172.21.15.133:0/1250719396 >> v2:172.21.15.133:3300/0 conn(0x7fe1b80a1010 0x7fe1b814bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bcda0640 1 --2- 172.21.15.133:0/1250719396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b806afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1affff640 1 -- 172.21.15.133:0/1250719396 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782767456 0 0) 0x7fe1b806a8f0 con 0x7fe1b80a2cf0 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1affff640 1 -- 172.21.15.133:0/1250719396 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1a4003530 con 0x7fe1b80a2cf0 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bdda2640 1 -- 172.21.15.133:0/1250719396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 msgr2=0x7fe1b806afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bdda2640 1 --2- 172.21.15.133:0/1250719396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b806afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bdda2640 1 -- 172.21.15.133:0/1250719396 >> v1:172.21.15.133:6789/0 conn(0x7fe1b80a2cf0 legacy=0x7fe1b80a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bdda2640 1 -- 172.21.15.133:0/1250719396 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b80a3950 con 0x7fe1b80a1010 2024-09-16T15:07:19.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bcda0640 1 --2- 172.21.15.133:0/1250719396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b806afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:19.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1bdda2640 1 --2- 172.21.15.133:0/1250719396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 0x7fe1b814bd90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40097c0 tx=0x7fe1b401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6db104994abae36 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:19.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1affff640 1 -- 172.21.15.133:0/1250719396 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40ad680 con 0x7fe1b80a1010 2024-09-16T15:07:19.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1affff640 1 -- 172.21.15.133:0/1250719396 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b40ad7e0 con 0x7fe1b80a1010 2024-09-16T15:07:19.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.750+0000 7fe1affff640 1 -- 172.21.15.133:0/1250719396 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40adb70 con 0x7fe1b80a1010 2024-09-16T15:07:19.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- 172.21.15.133:0/1250719396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 msgr2=0x7fe1b814bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 --2- 172.21.15.133:0/1250719396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 0x7fe1b814bd90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40097c0 tx=0x7fe1b401c360 comp rx=0 tx=0).stop 2024-09-16T15:07:19.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- 172.21.15.133:0/1250719396 shutdown_connections 2024-09-16T15:07:19.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 --2- 172.21.15.133:0/1250719396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 0x7fe1b814bd90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 --2- 172.21.15.133:0/1250719396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b806afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- 172.21.15.133:0/1250719396 >> 172.21.15.133:0/1250719396 conn(0x7fe1b80a3fd0 msgr2=0x7fe1b80a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:19.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- 172.21.15.133:0/1250719396 shutdown_connections 2024-09-16T15:07:19.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- 172.21.15.133:0/1250719396 wait complete. 2024-09-16T15:07:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 Processor -- start 2024-09-16T15:07:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- start start 2024-09-16T15:07:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b809ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 0x7fe1b8141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b80a2cf0 0x7fe1b806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b806e480 con 0x7fe1b80a1010 2024-09-16T15:07:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b806f2d0 con 0x7fe1b80a2cf0 2024-09-16T15:07:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8071be0 con 0x7fe1b809e490 2024-09-16T15:07:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bcda0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 0x7fe1b8141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bd5a1640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b809ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bcda0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 0x7fe1b8141d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51486/0 (socket says 172.21.15.133:51486) 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bd5a1640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b809ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:33996/0 (socket says 172.21.15.133:33996) 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bcda0640 1 -- 172.21.15.133:0/3605510606 learned_addr learned my addr 172.21.15.133:0/3605510606 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bdda2640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b80a2cf0 0x7fe1b806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bcda0640 1 -- 172.21.15.133:0/3605510606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 msgr2=0x7fe1b809ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bcda0640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b809ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bcda0640 1 -- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b80a2cf0 msgr2=0x7fe1b806dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bcda0640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b80a2cf0 0x7fe1b806dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bcda0640 1 -- 172.21.15.133:0/3605510606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a8009420 con 0x7fe1b80a1010 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bd5a1640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b809ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bdda2640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b80a2cf0 0x7fe1b806dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bcda0640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 0x7fe1b8141d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800ebe0 tx=0x7fe1a800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1adffb640 1 -- 172.21.15.133:0/3605510606 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800eee0 con 0x7fe1b80a1010 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1adffb640 1 -- 172.21.15.133:0/3605510606 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a800cc30 con 0x7fe1b80a1010 2024-09-16T15:07:19.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- 172.21.15.133:0/3605510606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b4009420 con 0x7fe1b80a1010 2024-09-16T15:07:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1adffb640 1 -- 172.21.15.133:0/3605510606 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8013470 con 0x7fe1b80a1010 2024-09-16T15:07:19.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- 172.21.15.133:0/3605510606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b809dc20 con 0x7fe1b80a1010 2024-09-16T15:07:19.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.754+0000 7fe1bf805640 1 -- 172.21.15.133:0/3605510606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b809e220 con 0x7fe1b80a1010 2024-09-16T15:07:19.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.758+0000 7fe1adffb640 1 -- 172.21.15.133:0/3605510606 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fe1a80135d0 con 0x7fe1b80a1010 2024-09-16T15:07:19.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.758+0000 7fe1adffb640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:19.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.758+0000 7fe1adffb640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:19.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.758+0000 7fe1adffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:19.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.758+0000 7fe1adffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:19.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.758+0000 7fe1adffb640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe19c097870 0x7fe19c099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:19.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.758+0000 7fe1adffb640 1 -- 172.21.15.133:0/3605510606 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3538+0+0 (secure 0 0 0) 0x7fe1a80c0460 con 0x7fe1b80a1010 2024-09-16T15:07:19.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.758+0000 7fe1bd5a1640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe19c097870 0x7fe19c099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:19.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.758+0000 7fe1bd5a1640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe19c097870 0x7fe19c099d30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0001570 tx=0x7fe1b0001a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:19.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.762+0000 7fe1adffb640 1 -- 172.21.15.133:0/3605510606 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe1a808dea0 con 0x7fe1b80a1010 2024-09-16T15:07:19.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.906+0000 7fe1bf805640 1 -- 172.21.15.133:0/3605510606 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe1b806afe0 con 0x7fe1b80a1010 2024-09-16T15:07:19.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.906+0000 7fe1adffb640 1 -- 172.21.15.133:0/3605510606 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe1a80210a0 con 0x7fe1b80a1010 2024-09-16T15:07:19.911 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 8 2024-09-16T15:07:19.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 10 mgrc shutdown 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 -- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe19c097870 msgr2=0x7fe19c099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe19c097870 0x7fe19c099d30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0001570 tx=0x7fe1b0001a00 comp rx=0 tx=0).stop 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 -- 172.21.15.133:0/3605510606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 msgr2=0x7fe1b8141d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 0x7fe1b8141d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800ebe0 tx=0x7fe1a800c520 comp rx=0 tx=0).stop 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 -- 172.21.15.133:0/3605510606 shutdown_connections 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1b80a2cf0 0x7fe1b806dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe19c097870 0x7fe19c099d30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1b80a1010 0x7fe1b8141d90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 --2- 172.21.15.133:0/3605510606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1b809e490 0x7fe1b809ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 -- 172.21.15.133:0/3605510606 >> 172.21.15.133:0/3605510606 conn(0x7fe1b80a3fd0 msgr2=0x7fe1b8143020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 -- 172.21.15.133:0/3605510606 shutdown_connections 2024-09-16T15:07:19.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:19.910+0000 7fe1bf805640 1 -- 172.21.15.133:0/3605510606 wait complete. 2024-09-16T15:07:19.928 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:07:20.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.026+0000 7fd8fb4ec640 1 Processor -- start 2024-09-16T15:07:20.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.026+0000 7fd8fb4ec640 1 -- start start 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd8f40a6210 0x7fd8f409edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8f40a2fe0 0x7fd8f4140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f409f310 con 0x7fd8f4141110 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f40a3590 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f40a3850 con 0x7fd8f40a6210 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8f8a87640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8f40a2fe0 0x7fd8f4140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8f8a87640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8f40a2fe0 0x7fd8f4140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51502/0 (socket says 172.21.15.133:51502) 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8f8a87640 1 -- 172.21.15.133:0/360029040 learned_addr learned my addr 172.21.15.133:0/360029040 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8f8a87640 1 -- 172.21.15.133:0/360029040 >> v2:172.21.15.191:3300/0 conn(0x7fd8f40a6210 msgr2=0x7fd8f409edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8f8a87640 1 --2- 172.21.15.133:0/360029040 >> v2:172.21.15.191:3300/0 conn(0x7fd8f40a6210 0x7fd8f409edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8f8a87640 1 -- 172.21.15.133:0/360029040 >> v1:172.21.15.191:6789/0 conn(0x7fd8f4141110 legacy=0x7fd8f4143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:20.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8f8a87640 1 -- 172.21.15.133:0/360029040 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f41415c0 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8f8a87640 1 --2- 172.21.15.133:0/360029040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 0x7fd8f4140bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd8f406a4d0 tx=0x7fd8f001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d963c9acea0131 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:20.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8ebfff640 1 -- 172.21.15.133:0/360029040 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00ab600 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8ebfff640 1 -- 172.21.15.133:0/360029040 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f00a7050 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8ebfff640 1 -- 172.21.15.133:0/360029040 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00ab980 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/360029040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 msgr2=0x7fd8f4140bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 --2- 172.21.15.133:0/360029040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 0x7fd8f4140bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd8f406a4d0 tx=0x7fd8f001c430 comp rx=0 tx=0).stop 2024-09-16T15:07:20.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/360029040 shutdown_connections 2024-09-16T15:07:20.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 --2- 172.21.15.133:0/360029040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 0x7fd8f4140bd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 --2- 172.21.15.133:0/360029040 >> v2:172.21.15.191:3300/0 conn(0x7fd8f40a6210 0x7fd8f409edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.030+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/360029040 >> 172.21.15.133:0/360029040 conn(0x7fd8f40a3fd0 msgr2=0x7fd8f40a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/360029040 shutdown_connections 2024-09-16T15:07:20.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/360029040 wait complete. 2024-09-16T15:07:20.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 Processor -- start 2024-09-16T15:07:20.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 -- start start 2024-09-16T15:07:20.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 0x7fd8f409d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8f40a6210 0x7fd8f4070f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8f4141110 0x7fd8f409b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f4098e60 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f4099100 con 0x7fd8f40a6210 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f40993c0 con 0x7fd8f4141110 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f9288640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 0x7fd8f409d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f9288640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 0x7fd8f409d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51512/0 (socket says 172.21.15.133:51512) 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f9288640 1 -- 172.21.15.133:0/2652435602 learned_addr learned my addr 172.21.15.133:0/2652435602 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f9a89640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8f4141110 0x7fd8f409b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f8a87640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8f40a6210 0x7fd8f4070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f9288640 1 -- 172.21.15.133:0/2652435602 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8f4141110 msgr2=0x7fd8f409b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f9288640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8f4141110 0x7fd8f409b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f9288640 1 -- 172.21.15.133:0/2652435602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8f40a6210 msgr2=0x7fd8f4070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f9288640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8f40a6210 0x7fd8f4070f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f9288640 1 -- 172.21.15.133:0/2652435602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f00094d0 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8f9288640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 0x7fd8f409d5c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400b550 tx=0x7fd8e400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:20.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8e9ffb640 1 -- 172.21.15.133:0/2652435602 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4010870 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8e9ffb640 1 -- 172.21.15.133:0/2652435602 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e40109f0 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/2652435602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f40a43b0 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8e9ffb640 1 -- 172.21.15.133:0/2652435602 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4010da0 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/2652435602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f40714a0 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.034+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/2652435602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f4140bd0 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.038+0000 7fd8e9ffb640 1 -- 172.21.15.133:0/2652435602 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fd8e4023480 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.038+0000 7fd8e9ffb640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:20.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.038+0000 7fd8e9ffb640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:20.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.038+0000 7fd8e9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:20.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.038+0000 7fd8e9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:20.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.038+0000 7fd8e9ffb640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd8d80978c0 0x7fd8d8099d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.038+0000 7fd8e9ffb640 1 -- 172.21.15.133:0/2652435602 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3538+0+0 (secure 0 0 0) 0x7fd8e40bfe10 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.038+0000 7fd8f8a87640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd8d80978c0 0x7fd8d8099d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.038+0000 7fd8f8a87640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd8d80978c0 0x7fd8d8099d80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00ad000 tx=0x7fd8f0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:20.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.042+0000 7fd8e9ffb640 1 -- 172.21.15.133:0/2652435602 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd8e408d840 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.190+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/2652435602 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd8f40a19a0 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.190+0000 7fd8e9ffb640 1 -- 172.21.15.133:0/2652435602 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd8e401e090 con 0x7fd8f40a2fe0 2024-09-16T15:07:20.196 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:20.196 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-09-16T15:07:20.196 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-16T15:07:20.196 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-09-16T15:07:20.196 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-09-16T15:07:20.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-09-16T15:07:20.196 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-09-16T15:07:20.196 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 2, 2024-09-16T15:07:20.196 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": "rbd" 2024-09-16T15:07:20.196 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-09-16T15:07:20.197 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-09-16T15:07:20.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 10 mgrc shutdown 2024-09-16T15:07:20.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/2652435602 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd8d80978c0 msgr2=0x7fd8d8099d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd8d80978c0 0x7fd8d8099d80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00ad000 tx=0x7fd8f0008040 comp rx=0 tx=0).stop 2024-09-16T15:07:20.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/2652435602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 msgr2=0x7fd8f409d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 0x7fd8f409d5c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400b550 tx=0x7fd8e400ba10 comp rx=0 tx=0).stop 2024-09-16T15:07:20.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/2652435602 shutdown_connections 2024-09-16T15:07:20.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8f4141110 0x7fd8f409b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8f40a6210 0x7fd8f4070f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd8d80978c0 0x7fd8d8099d80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 --2- 172.21.15.133:0/2652435602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8f40a2fe0 0x7fd8f409d5c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/2652435602 >> 172.21.15.133:0/2652435602 conn(0x7fd8f40a3fd0 msgr2=0x7fd8f406b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:20.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/2652435602 shutdown_connections 2024-09-16T15:07:20.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.194+0000 7fd8fb4ec640 1 -- 172.21.15.133:0/2652435602 wait complete. 2024-09-16T15:07:20.213 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:07:20.213 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:07:20.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.322+0000 7fd51f111640 1 Processor -- start 2024-09-16T15:07:20.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.322+0000 7fd51f111640 1 -- start start 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.322+0000 7fd51f111640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd518065f60 0x7fd5180a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51f111640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd518066340 con 0x7fd5180a7700 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51f111640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5180a2980 con 0x7fd51809f1b0 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51f111640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5180a2c40 con 0x7fd518065f60 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd517fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd5180a7700 0x7fd51809eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52468/0 (socket says 172.21.15.133:52468) 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd517fff640 1 -- 172.21.15.133:0/4061619745 learned_addr learned my addr 172.21.15.133:0/4061619745 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51cead640 1 --2- 172.21.15.133:0/4061619745 >> v2:172.21.15.191:3300/0 conn(0x7fd518065f60 0x7fd5180a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440153197 0 0) 0x7fd518066340 con 0x7fd5180a7700 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd508003530 con 0x7fd5180a7700 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1098060735 0 0) 0x7fd508003530 con 0x7fd5180a7700 2024-09-16T15:07:20.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 >> v2:172.21.15.191:3300/0 conn(0x7fd518065f60 msgr2=0x7fd5180a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 --2- 172.21.15.133:0/4061619745 >> v2:172.21.15.191:3300/0 conn(0x7fd518065f60 0x7fd5180a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 >> v1:172.21.15.191:6789/0 conn(0x7fd51809f1b0 legacy=0x7fd51814a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180a6a00 con 0x7fd5180a7700 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7fd504001680 con 0x7fd5180a7700 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 >> v1:172.21.15.133:6789/0 conn(0x7fd5180a7700 legacy=0x7fd51809eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508003980 0x7fd508003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5080047d0 0x7fd508004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd508005190 0x7fd50800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180a2c40 con 0x7fd508005190 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50800c130 con 0x7fd5080047d0 2024-09-16T15:07:20.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50800c2f0 con 0x7fd508003980 2024-09-16T15:07:20.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51cead640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508003980 0x7fd508003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51d6ae640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd508005190 0x7fd50800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd517fff640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5080047d0 0x7fd508004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51d6ae640 1 -- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508003980 msgr2=0x7fd508003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:20.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51d6ae640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508003980 0x7fd508003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51d6ae640 1 -- 172.21.15.133:0/4061619745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5080047d0 msgr2=0x7fd508004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:20.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51d6ae640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5080047d0 0x7fd508004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51d6ae640 1 -- 172.21.15.133:0/4061619745 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd508003530 con 0x7fd508005190 2024-09-16T15:07:20.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51cead640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508003980 0x7fd508003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:20.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51d6ae640 1 -- 172.21.15.133:0/4061619745 reap_dead start 2024-09-16T15:07:20.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd51d6ae640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd508005190 0x7fd50800b990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd510001e70 tx=0x7fd51001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec670b119076ec2f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:20.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510003b80 con 0x7fd508005190 2024-09-16T15:07:20.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510003d20 con 0x7fd508005190 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.326+0000 7fd5177fe640 1 -- 172.21.15.133:0/4061619745 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510005310 con 0x7fd508005190 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd508005190 msgr2=0x7fd50800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd508005190 0x7fd50800b990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd510001e70 tx=0x7fd51001c760 comp rx=0 tx=0).stop 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- 172.21.15.133:0/4061619745 shutdown_connections 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 --2- 172.21.15.133:0/4061619745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd508005190 0x7fd50800b990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- 172.21.15.133:0/4061619745 >> 172.21.15.133:0/4061619745 conn(0x7fd5180a8030 msgr2=0x7fd5180a45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- 172.21.15.133:0/4061619745 shutdown_connections 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- 172.21.15.133:0/4061619745 wait complete. 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 Processor -- start 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- start start 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508005190 0x7fd518071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd51809e5b0 0x7fd51809ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd51809d390 0x7fd51809d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180721d0 con 0x7fd51809d390 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180a8d60 con 0x7fd51809e5b0 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180a9020 con 0x7fd508005190 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51cead640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508005190 0x7fd518071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51cead640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508005190 0x7fd518071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:34024/0 (socket says 172.21.15.133:34024) 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51cead640 1 -- 172.21.15.133:0/940798438 learned_addr learned my addr 172.21.15.133:0/940798438 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51d6ae640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd51809d390 0x7fd51809d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd517fff640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd51809e5b0 0x7fd51809ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51cead640 1 -- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd51809e5b0 msgr2=0x7fd51809ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51cead640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd51809e5b0 0x7fd51809ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51cead640 1 -- 172.21.15.133:0/940798438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd51809d390 msgr2=0x7fd51809d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51cead640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd51809d390 0x7fd51809d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51cead640 1 -- 172.21.15.133:0/940798438 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd518142e70 con 0x7fd508005190 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51d6ae640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd51809d390 0x7fd51809d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd517fff640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd51809e5b0 0x7fd51809ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51cead640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508005190 0x7fd518071610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd50c011ac0 tx=0x7fd50c011f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd5157fa640 1 -- 172.21.15.133:0/940798438 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c00fc10 con 0x7fd508005190 2024-09-16T15:07:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd5157fa640 1 -- 172.21.15.133:0/940798438 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c00fd70 con 0x7fd508005190 2024-09-16T15:07:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- 172.21.15.133:0/940798438 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd518143150 con 0x7fd508005190 2024-09-16T15:07:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd5157fa640 1 -- 172.21.15.133:0/940798438 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c00fee0 con 0x7fd508005190 2024-09-16T15:07:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- 172.21.15.133:0/940798438 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd518141f00 con 0x7fd508005190 2024-09-16T15:07:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.330+0000 7fd51f111640 1 -- 172.21.15.133:0/940798438 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5180672a0 con 0x7fd508005190 2024-09-16T15:07:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.334+0000 7fd5157fa640 1 -- 172.21.15.133:0/940798438 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fd50c004240 con 0x7fd508005190 2024-09-16T15:07:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.334+0000 7fd5157fa640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.334+0000 7fd5157fa640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.334+0000 7fd5157fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.334+0000 7fd5157fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.334+0000 7fd5157fa640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd4fc097840 0x7fd4fc099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.334+0000 7fd5157fa640 1 -- 172.21.15.133:0/940798438 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3538+0+0 (secure 0 0 0) 0x7fd50c0c04a0 con 0x7fd508005190 2024-09-16T15:07:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.334+0000 7fd517fff640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd4fc097840 0x7fd4fc099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.334+0000 7fd517fff640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd4fc097840 0x7fd4fc099d00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5040017b0 tx=0x7fd504000ee0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:20.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.338+0000 7fd5157fa640 1 -- 172.21.15.133:0/940798438 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd50c08df70 con 0x7fd508005190 2024-09-16T15:07:20.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.482+0000 7fd51f111640 1 -- 172.21.15.133:0/940798438 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd518072310 con 0x7fd508005190 2024-09-16T15:07:20.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.670+0000 7fd5157fa640 1 -- 172.21.15.133:0/940798438 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd50c022030 con 0x7fd508005190 2024-09-16T15:07:20.675 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-09-16T15:07:20.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 10 mgrc shutdown 2024-09-16T15:07:20.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 -- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd4fc097840 msgr2=0x7fd4fc099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd4fc097840 0x7fd4fc099d00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5040017b0 tx=0x7fd504000ee0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 -- 172.21.15.133:0/940798438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508005190 msgr2=0x7fd518071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508005190 0x7fd518071610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd50c011ac0 tx=0x7fd50c011f80 comp rx=0 tx=0).stop 2024-09-16T15:07:20.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 -- 172.21.15.133:0/940798438 shutdown_connections 2024-09-16T15:07:20.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd4fc097840 0x7fd4fc099d00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd51809d390 0x7fd51809d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd51809e5b0 0x7fd51809ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 --2- 172.21.15.133:0/940798438 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd508005190 0x7fd518071610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 -- 172.21.15.133:0/940798438 >> 172.21.15.133:0/940798438 conn(0x7fd5180a8030 msgr2=0x7fd51806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:20.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 -- 172.21.15.133:0/940798438 shutdown_connections 2024-09-16T15:07:20.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.674+0000 7fd51f111640 1 -- 172.21.15.133:0/940798438 wait complete. 2024-09-16T15:07:20.699 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:07:20.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.822+0000 7fb7258cb640 1 Processor -- start 2024-09-16T15:07:20.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.822+0000 7fb7258cb640 1 -- start start 2024-09-16T15:07:20.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb7258cb640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb72006ca20 0x7fb7200a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb7258cb640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72009fa20 con 0x7fb72009f2a0 2024-09-16T15:07:20.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb7258cb640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7200a3cb0 con 0x7fb7200a38d0 2024-09-16T15:07:20.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb7258cb640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7200a3e40 con 0x7fb72006ca20 2024-09-16T15:07:20.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb72009f2a0 0x7fb72014bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52478/0 (socket says 172.21.15.133:52478) 2024-09-16T15:07:20.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71f7fe640 1 -- 172.21.15.133:0/1813258391 learned_addr learned my addr 172.21.15.133:0/1813258391 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:20.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb717fff640 1 --2- 172.21.15.133:0/1813258391 >> v2:172.21.15.191:3300/0 conn(0x7fb72006ca20 0x7fb7200a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690370310 0 0) 0x7fb72009fa20 con 0x7fb72009f2a0 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb70c003530 con 0x7fb72009f2a0 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2004506221 0 0) 0x7fb70c003530 con 0x7fb72009f2a0 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 >> v2:172.21.15.191:3300/0 conn(0x7fb72006ca20 msgr2=0x7fb7200a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 --2- 172.21.15.133:0/1813258391 >> v2:172.21.15.191:3300/0 conn(0x7fb72006ca20 0x7fb7200a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 >> v1:172.21.15.191:6789/0 conn(0x7fb7200a38d0 legacy=0x7fb720141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7201423d0 con 0x7fb72009f2a0 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7fb718001a00 con 0x7fb72009f2a0 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 >> v1:172.21.15.133:6789/0 conn(0x7fb72009f2a0 legacy=0x7fb72014bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb70c003980 0x7fb70c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c0047d0 0x7fb70c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb70c005190 0x7fb70c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c00c100 con 0x7fb70c003980 2024-09-16T15:07:20.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c00c270 con 0x7fb70c0047d0 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c00c430 con 0x7fb70c005190 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71effd640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb70c003980 0x7fb70c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71f7fe640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb70c005190 0x7fb70c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb717fff640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c0047d0 0x7fb70c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71effd640 1 -- 172.21.15.133:0/1813258391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb70c005190 msgr2=0x7fb70c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71effd640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb70c005190 0x7fb70c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71effd640 1 -- 172.21.15.133:0/1813258391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c0047d0 msgr2=0x7fb70c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71effd640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c0047d0 0x7fb70c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71effd640 1 -- 172.21.15.133:0/1813258391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c003530 con 0x7fb70c003980 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb717fff640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c0047d0 0x7fb70c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:07:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.826+0000 7fb71f7fe640 1 -- 172.21.15.133:0/1813258391 reap_dead start 2024-09-16T15:07:20.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb71effd640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb70c003980 0x7fb70c003da0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb710001780 tx=0x7fb71001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59059865989b5fc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:20.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7100052d0 con 0x7fb70c003980 2024-09-16T15:07:20.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7100a7050 con 0x7fb70c003980 2024-09-16T15:07:20.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb71e7fc640 1 -- 172.21.15.133:0/1813258391 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb710003600 con 0x7fb70c003980 2024-09-16T15:07:20.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 -- 172.21.15.133:0/1813258391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb70c003980 msgr2=0x7fb70c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb70c003980 0x7fb70c003da0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb710001780 tx=0x7fb71001c490 comp rx=0 tx=0).stop 2024-09-16T15:07:20.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 -- 172.21.15.133:0/1813258391 shutdown_connections 2024-09-16T15:07:20.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 --2- 172.21.15.133:0/1813258391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb70c003980 0x7fb70c003da0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 -- 172.21.15.133:0/1813258391 >> 172.21.15.133:0/1813258391 conn(0x7fb7200a88e0 msgr2=0x7fb7200a8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:20.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 -- 172.21.15.133:0/1813258391 shutdown_connections 2024-09-16T15:07:20.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 -- 172.21.15.133:0/1813258391 wait complete. 2024-09-16T15:07:20.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 Processor -- start 2024-09-16T15:07:20.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 -- start start 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c003980 0x7fb7200b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7200a7170 0x7fb72009ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7200a75b0 0x7fb72009d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72009e4a0 con 0x7fb7200a7170 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72009e740 con 0x7fb70c003980 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb7258cb640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72009ea00 con 0x7fb7200a75b0 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb717fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7200a7170 0x7fb72009ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb717fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7200a7170 0x7fb72009ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51544/0 (socket says 172.21.15.133:51544) 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb71f7fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7200a75b0 0x7fb72009d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb717fff640 1 -- 172.21.15.133:0/3098269147 learned_addr learned my addr 172.21.15.133:0/3098269147 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb717fff640 1 -- 172.21.15.133:0/3098269147 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7200a75b0 msgr2=0x7fb72009d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb717fff640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7200a75b0 0x7fb72009d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb71effd640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c003980 0x7fb7200b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb717fff640 1 -- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c003980 msgr2=0x7fb7200b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb717fff640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c003980 0x7fb7200b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb717fff640 1 -- 172.21.15.133:0/3098269147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7201d1d50 con 0x7fb7200a7170 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.830+0000 7fb71effd640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c003980 0x7fb7200b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb717fff640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7200a7170 0x7fb72009ce50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb708011510 tx=0x7fb7080119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb71cff9640 1 -- 172.21.15.133:0/3098269147 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708004220 con 0x7fb7200a7170 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb71cff9640 1 -- 172.21.15.133:0/3098269147 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb708004380 con 0x7fb7200a7170 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb7258cb640 1 -- 172.21.15.133:0/3098269147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7200a7c70 con 0x7fb7200a7170 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb71cff9640 1 -- 172.21.15.133:0/3098269147 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708015890 con 0x7fb7200a7170 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb7258cb640 1 -- 172.21.15.133:0/3098269147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7200b4b60 con 0x7fb7200a7170 2024-09-16T15:07:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb7258cb640 1 -- 172.21.15.133:0/3098269147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7200b5640 con 0x7fb7200a7170 2024-09-16T15:07:20.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb71cff9640 1 -- 172.21.15.133:0/3098269147 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fb7080159f0 con 0x7fb7200a7170 2024-09-16T15:07:20.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb71cff9640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:20.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb71cff9640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:20.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb71cff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:20.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb71cff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:20.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb71cff9640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fb700097840 0x7fb700099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:20.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.834+0000 7fb71cff9640 1 -- 172.21.15.133:0/3098269147 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3880+0+0 (secure 0 0 0) 0x7fb7080c0280 con 0x7fb7200a7170 2024-09-16T15:07:20.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.838+0000 7fb71cff9640 1 -- 172.21.15.133:0/3098269147 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb70808dc00 con 0x7fb7200a7170 2024-09-16T15:07:20.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.854+0000 7fb71effd640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fb700097840 0x7fb700099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:20.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.858+0000 7fb71effd640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fb700097840 0x7fb700099d00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb710001750 tx=0x7fb7100011a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:21.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:20.998+0000 7fb7258cb640 1 -- 172.21.15.133:0/3098269147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb72006a8e0 con 0x7fb7200a7170 2024-09-16T15:07:21.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.674+0000 7fb71cff9640 1 -- 172.21.15.133:0/3098269147 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb7080220d0 con 0x7fb7200a7170 2024-09-16T15:07:21.682 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-09-16T15:07:21.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 10 mgrc shutdown 2024-09-16T15:07:21.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 -- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fb700097840 msgr2=0x7fb700099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:21.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fb700097840 0x7fb700099d00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb710001750 tx=0x7fb7100011a0 comp rx=0 tx=0).stop 2024-09-16T15:07:21.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 -- 172.21.15.133:0/3098269147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7200a7170 msgr2=0x7fb72009ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:21.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7200a7170 0x7fb72009ce50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb708011510 tx=0x7fb7080119d0 comp rx=0 tx=0).stop 2024-09-16T15:07:21.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 -- 172.21.15.133:0/3098269147 shutdown_connections 2024-09-16T15:07:21.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7200a75b0 0x7fb72009d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:21.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fb700097840 0x7fb700099d00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:21.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7200a7170 0x7fb72009ce50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:21.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 --2- 172.21.15.133:0/3098269147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb70c003980 0x7fb7200b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:21.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 -- 172.21.15.133:0/3098269147 >> 172.21.15.133:0/3098269147 conn(0x7fb7200a88e0 msgr2=0x7fb7200a0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:21.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 -- 172.21.15.133:0/3098269147 shutdown_connections 2024-09-16T15:07:21.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.682+0000 7fb7258cb640 1 -- 172.21.15.133:0/3098269147 wait complete. 2024-09-16T15:07:21.708 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:07:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe645d13640 1 Processor -- start 2024-09-16T15:07:21.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe645d13640 1 -- start start 2024-09-16T15:07:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe645d13640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 0x7fe64009edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe645d13640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64009f310 con 0x7fe6400a6210 2024-09-16T15:07:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe645d13640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6400a3590 con 0x7fe6401412e0 2024-09-16T15:07:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe645d13640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400a3850 con 0x7fe6400a2fe0 2024-09-16T15:07:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63ffff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 0x7fe64009edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe6400a6210 0x7fe640140bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52482/0 (socket says 172.21.15.133:52482) 2024-09-16T15:07:21.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63f7fe640 1 -- 172.21.15.133:0/465454629 learned_addr learned my addr 172.21.15.133:0/465454629 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63ffff640 1 -- 172.21.15.133:0/465454629 >> v1:172.21.15.191:6789/0 conn(0x7fe6401412e0 legacy=0x7fe640148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63e7fc640 1 -- 172.21.15.133:0/465454629 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678806225 0 0) 0x7fe6300014a0 con 0x7fe6400a6210 2024-09-16T15:07:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63ffff640 1 -- 172.21.15.133:0/465454629 >> v1:172.21.15.133:6789/0 conn(0x7fe6400a6210 legacy=0x7fe640140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:21.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63ffff640 1 -- 172.21.15.133:0/465454629 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6400a65f0 con 0x7fe6400a2fe0 2024-09-16T15:07:21.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63ffff640 1 --2- 172.21.15.133:0/465454629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 0x7fe64009edd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe6380097f0 tx=0x7fe63801c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce62520df896d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:21.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63e7fc640 1 -- 172.21.15.133:0/465454629 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380ab600 con 0x7fe6400a2fe0 2024-09-16T15:07:21.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63e7fc640 1 -- 172.21.15.133:0/465454629 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6380a7050 con 0x7fe6400a2fe0 2024-09-16T15:07:21.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.834+0000 7fe63e7fc640 1 -- 172.21.15.133:0/465454629 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380ab980 con 0x7fe6400a2fe0 2024-09-16T15:07:21.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- 172.21.15.133:0/465454629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 msgr2=0x7fe64009edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:21.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 --2- 172.21.15.133:0/465454629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 0x7fe64009edd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe6380097f0 tx=0x7fe63801c360 comp rx=0 tx=0).stop 2024-09-16T15:07:21.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- 172.21.15.133:0/465454629 shutdown_connections 2024-09-16T15:07:21.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 --2- 172.21.15.133:0/465454629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 0x7fe64009edd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:21.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- 172.21.15.133:0/465454629 >> 172.21.15.133:0/465454629 conn(0x7fe6400a3fd0 msgr2=0x7fe6400a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:21.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- 172.21.15.133:0/465454629 shutdown_connections 2024-09-16T15:07:21.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- 172.21.15.133:0/465454629 wait complete. 2024-09-16T15:07:21.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 Processor -- start 2024-09-16T15:07:21.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- start start 2024-09-16T15:07:21.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 0x7fe640070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6400a6210 0x7fe6400b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401412e0 0x7fe6400b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400b55a0 con 0x7fe6401412e0 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400b14f0 con 0x7fe6400a6210 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400b17b0 con 0x7fe6400a2fe0 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401412e0 0x7fe6400b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401412e0 0x7fe6400b5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51556/0 (socket says 172.21.15.133:51556) 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63ffff640 1 -- 172.21.15.133:0/2472479407 learned_addr learned my addr 172.21.15.133:0/2472479407 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63f7fe640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 0x7fe640070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63effd640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6400a6210 0x7fe6400b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63ffff640 1 -- 172.21.15.133:0/2472479407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 msgr2=0x7fe640070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:21.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63ffff640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 0x7fe640070420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:21.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63ffff640 1 -- 172.21.15.133:0/2472479407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6400a6210 msgr2=0x7fe6400b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:21.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63ffff640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6400a6210 0x7fe6400b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:21.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63ffff640 1 -- 172.21.15.133:0/2472479407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe638009420 con 0x7fe6401412e0 2024-09-16T15:07:21.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe63ffff640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401412e0 0x7fe6400b5060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe63801aa10 tx=0x7fe63801aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:21.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe627fff640 1 -- 172.21.15.133:0/2472479407 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638004230 con 0x7fe6401412e0 2024-09-16T15:07:21.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe627fff640 1 -- 172.21.15.133:0/2472479407 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe638004390 con 0x7fe6401412e0 2024-09-16T15:07:21.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe627fff640 1 -- 172.21.15.133:0/2472479407 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638003840 con 0x7fe6401412e0 2024-09-16T15:07:21.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- 172.21.15.133:0/2472479407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6400a4440 con 0x7fe6401412e0 2024-09-16T15:07:21.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.838+0000 7fe645d13640 1 -- 172.21.15.133:0/2472479407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6400b2310 con 0x7fe6401412e0 2024-09-16T15:07:21.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.842+0000 7fe645d13640 1 -- 172.21.15.133:0/2472479407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64009edd0 con 0x7fe6401412e0 2024-09-16T15:07:21.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.842+0000 7fe627fff640 1 -- 172.21.15.133:0/2472479407 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fe6380039a0 con 0x7fe6401412e0 2024-09-16T15:07:21.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.842+0000 7fe627fff640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:21.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.842+0000 7fe627fff640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:21.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.842+0000 7fe627fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:21.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.842+0000 7fe627fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:21.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.842+0000 7fe627fff640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe62c097870 0x7fe62c099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:21.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.842+0000 7fe627fff640 1 -- 172.21.15.133:0/2472479407 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4262+0+0 (secure 0 0 0) 0x7fe63815aa30 con 0x7fe6401412e0 2024-09-16T15:07:21.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.842+0000 7fe63f7fe640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe62c097870 0x7fe62c099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:21.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.842+0000 7fe63f7fe640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe62c097870 0x7fe62c099d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe6300012f0 tx=0x7fe630007f20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:21.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:21.846+0000 7fe627fff640 1 -- 172.21.15.133:0/2472479407 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe6381281a0 con 0x7fe6401412e0 2024-09-16T15:07:22.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.042+0000 7fe645d13640 1 -- 172.21.15.133:0/2472479407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe640142a00 con 0x7fe6401412e0 2024-09-16T15:07:22.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.050+0000 7fe627fff640 1 -- 172.21.15.133:0/2472479407 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe6380c40c0 con 0x7fe6401412e0 2024-09-16T15:07:22.056 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 3 and data pool 4 2024-09-16T15:07:22.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 10 mgrc shutdown 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 -- 172.21.15.133:0/2472479407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe62c097870 msgr2=0x7fe62c099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe62c097870 0x7fe62c099d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe6300012f0 tx=0x7fe630007f20 comp rx=0 tx=0).stop 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 -- 172.21.15.133:0/2472479407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401412e0 msgr2=0x7fe6400b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401412e0 0x7fe6400b5060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe63801aa10 tx=0x7fe63801aba0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 -- 172.21.15.133:0/2472479407 shutdown_connections 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe62c097870 0x7fe62c099d30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6401412e0 0x7fe6400b5060 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6400a6210 0x7fe6400b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 --2- 172.21.15.133:0/2472479407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6400a2fe0 0x7fe640070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 -- 172.21.15.133:0/2472479407 >> 172.21.15.133:0/2472479407 conn(0x7fe6400a3fd0 msgr2=0x7fe6400a2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 -- 172.21.15.133:0/2472479407 shutdown_connections 2024-09-16T15:07:22.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.054+0000 7fe645d13640 1 -- 172.21.15.133:0/2472479407 wait complete. 2024-09-16T15:07:22.081 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:22.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.190+0000 7f8c3a0a0640 1 Processor -- start 2024-09-16T15:07:22.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.190+0000 7f8c3a0a0640 1 -- start start 2024-09-16T15:07:22.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c3a0a0640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c340a2fe0 0x7f8c3409edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c3a0a0640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3409f310 con 0x7f8c340a6210 2024-09-16T15:07:22.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c3a0a0640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c340a3590 con 0x7f8c341412e0 2024-09-16T15:07:22.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c3a0a0640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c340a3850 con 0x7f8c340a2fe0 2024-09-16T15:07:22.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c337fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8c340a6210 0x7f8c34140bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52498/0 (socket says 172.21.15.133:52498) 2024-09-16T15:07:22.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c337fe640 1 -- 172.21.15.133:0/3125614134 learned_addr learned my addr 172.21.15.133:0/3125614134 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:22.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c33fff640 1 --2- 172.21.15.133:0/3125614134 >> v2:172.21.15.133:3300/0 conn(0x7f8c340a2fe0 0x7f8c3409edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c33fff640 1 -- 172.21.15.133:0/3125614134 >> v1:172.21.15.191:6789/0 conn(0x7f8c341412e0 legacy=0x7f8c34148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:22.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c33fff640 1 -- 172.21.15.133:0/3125614134 >> v1:172.21.15.133:6789/0 conn(0x7f8c340a6210 legacy=0x7f8c34140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c32ffd640 1 -- 172.21.15.133:0/3125614134 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467250736 0 0) 0x7f8c1c001120 con 0x7f8c340a6210 2024-09-16T15:07:22.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c33fff640 1 -- 172.21.15.133:0/3125614134 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c340a65f0 con 0x7f8c340a2fe0 2024-09-16T15:07:22.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c33fff640 1 --2- 172.21.15.133:0/3125614134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a2fe0 0x7f8c3409edd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009d90 tx=0x7f8c2c01c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1395d3a2d49bebdc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:22.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c32ffd640 1 -- 172.21.15.133:0/3125614134 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c0a57c0 con 0x7f8c340a2fe0 2024-09-16T15:07:22.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c32ffd640 1 -- 172.21.15.133:0/3125614134 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c0a5920 con 0x7f8c340a2fe0 2024-09-16T15:07:22.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.194+0000 7f8c32ffd640 1 -- 172.21.15.133:0/3125614134 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c0a5cb0 con 0x7f8c340a2fe0 2024-09-16T15:07:22.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/3125614134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a2fe0 msgr2=0x7f8c3409edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 --2- 172.21.15.133:0/3125614134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a2fe0 0x7f8c3409edd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009d90 tx=0x7f8c2c01c7f0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/3125614134 shutdown_connections 2024-09-16T15:07:22.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 --2- 172.21.15.133:0/3125614134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a2fe0 0x7f8c3409edd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/3125614134 >> 172.21.15.133:0/3125614134 conn(0x7f8c340a3fd0 msgr2=0x7f8c340a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:22.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/3125614134 shutdown_connections 2024-09-16T15:07:22.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/3125614134 wait complete. 2024-09-16T15:07:22.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 Processor -- start 2024-09-16T15:07:22.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- start start 2024-09-16T15:07:22.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c340a2fe0 0x7f8c341422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a6210 0x7f8c3413e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c341412e0 0x7f8c34135db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c34137210 con 0x7f8c340a6210 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3413e460 con 0x7f8c341412e0 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3413fe40 con 0x7f8c340a2fe0 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c337fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c340a2fe0 0x7f8c341422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c2bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a6210 0x7f8c3413e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c2bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a6210 0x7f8c3413e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51588/0 (socket says 172.21.15.133:51588) 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c2bfff640 1 -- 172.21.15.133:0/2721310855 learned_addr learned my addr 172.21.15.133:0/2721310855 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c33fff640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c341412e0 0x7f8c34135db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c2bfff640 1 -- 172.21.15.133:0/2721310855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c340a2fe0 msgr2=0x7f8c341422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c2bfff640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c340a2fe0 0x7f8c341422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c2bfff640 1 -- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c341412e0 msgr2=0x7f8c34135db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c2bfff640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c341412e0 0x7f8c34135db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c2bfff640 1 -- 172.21.15.133:0/2721310855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c009a70 con 0x7f8c340a6210 2024-09-16T15:07:22.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c33fff640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c341412e0 0x7f8c34135db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:22.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c2bfff640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a6210 0x7f8c3413e730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400b3e0 tx=0x7f8c2400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:22.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c30ff9640 1 -- 172.21.15.133:0/2721310855 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24011830 con 0x7f8c340a6210 2024-09-16T15:07:22.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c30ff9640 1 -- 172.21.15.133:0/2721310855 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c24011990 con 0x7f8c340a6210 2024-09-16T15:07:22.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/2721310855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c34140100 con 0x7f8c340a6210 2024-09-16T15:07:22.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c30ff9640 1 -- 172.21.15.133:0/2721310855 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24011d80 con 0x7f8c340a6210 2024-09-16T15:07:22.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/2721310855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3413f590 con 0x7f8c340a6210 2024-09-16T15:07:22.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.198+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/2721310855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c34136f30 con 0x7f8c340a6210 2024-09-16T15:07:22.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.202+0000 7f8c30ff9640 1 -- 172.21.15.133:0/2721310855 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f8c24004020 con 0x7f8c340a6210 2024-09-16T15:07:22.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.202+0000 7f8c30ff9640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:22.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.202+0000 7f8c30ff9640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:22.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.202+0000 7f8c30ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:22.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.202+0000 7f8c30ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:22.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.202+0000 7f8c30ff9640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8c14097870 0x7f8c14099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.202+0000 7f8c30ff9640 1 -- 172.21.15.133:0/2721310855 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f8c240bff70 con 0x7f8c340a6210 2024-09-16T15:07:22.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.202+0000 7f8c337fe640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8c14097870 0x7f8c14099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.202+0000 7f8c337fe640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8c14097870 0x7f8c14099d30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c000f70 tx=0x7f8c1c007ba0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:22.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.206+0000 7f8c30ff9640 1 -- 172.21.15.133:0/2721310855 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8c2408d680 con 0x7f8c340a6210 2024-09-16T15:07:22.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.362+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/2721310855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c340671f0 con 0x7f8c340a6210 2024-09-16T15:07:22.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.362+0000 7f8c30ff9640 1 -- 172.21.15.133:0/2721310855 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13454 (secure 0 0 0) 0x7f8c24015090 con 0x7f8c340a6210 2024-09-16T15:07:22.368 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:22.369 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":20,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:22.047703+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:07:20.523939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-16T15:07:21.043356+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 10 mgrc shutdown 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8c14097870 msgr2=0x7f8c14099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8c14097870 0x7f8c14099d30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c000f70 tx=0x7f8c1c007ba0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/2721310855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a6210 msgr2=0x7f8c3413e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a6210 0x7f8c3413e730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400b3e0 tx=0x7f8c2400b8a0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/2721310855 shutdown_connections 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c341412e0 0x7f8c34135db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f8c14097870 0x7f8c14099d30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c340a6210 0x7f8c3413e730 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 --2- 172.21.15.133:0/2721310855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c340a2fe0 0x7f8c341422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/2721310855 >> 172.21.15.133:0/2721310855 conn(0x7f8c340a3fd0 msgr2=0x7f8c34143bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/2721310855 shutdown_connections 2024-09-16T15:07:22.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.366+0000 7f8c3a0a0640 1 -- 172.21.15.133:0/2721310855 wait complete. 2024-09-16T15:07:22.386 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:22.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.498+0000 7fdd63f31640 1 Processor -- start 2024-09-16T15:07:22.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.498+0000 7fdd63f31640 1 -- start start 2024-09-16T15:07:22.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd63f31640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c0a48d0 0x7fdd5c09e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd63f31640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdd5c13e530 0x7fdd5c14bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd63f31640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c0a4cb0 con 0x7fdd5c09ff90 2024-09-16T15:07:22.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd63f31640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c09eaf0 con 0x7fdd5c13e530 2024-09-16T15:07:22.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd63f31640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c09fb00 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd61ccd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c0a48d0 0x7fdd5c09e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd614cc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdd5c09ff90 0x7fdd5c141140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52506/0 (socket says 172.21.15.133:52506) 2024-09-16T15:07:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd614cc640 1 -- 172.21.15.133:0/106762848 learned_addr learned my addr 172.21.15.133:0/106762848 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd624ce640 1 --2- 172.21.15.133:0/106762848 >> v2:172.21.15.191:3300/0 conn(0x7fdd5c13e530 0x7fdd5c14bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd61ccd640 1 -- 172.21.15.133:0/106762848 >> v2:172.21.15.191:3300/0 conn(0x7fdd5c13e530 msgr2=0x7fdd5c14bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd60ccb640 1 -- 172.21.15.133:0/106762848 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612264794 0 0) 0x7fdd5c0a4cb0 con 0x7fdd5c09ff90 2024-09-16T15:07:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd61ccd640 1 --2- 172.21.15.133:0/106762848 >> v2:172.21.15.191:3300/0 conn(0x7fdd5c13e530 0x7fdd5c14bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd61ccd640 1 -- 172.21.15.133:0/106762848 >> v1:172.21.15.133:6789/0 conn(0x7fdd5c09ff90 legacy=0x7fdd5c141140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd61ccd640 1 -- 172.21.15.133:0/106762848 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c1419d0 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd624ce640 1 --2- 172.21.15.133:0/106762848 >> v2:172.21.15.191:3300/0 conn(0x7fdd5c13e530 0x7fdd5c14bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:22.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd61ccd640 1 --2- 172.21.15.133:0/106762848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c0a48d0 0x7fdd5c09e5b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c141bc0 tx=0x7fdd5801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4bcd0983d6d686 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:22.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd60ccb640 1 -- 172.21.15.133:0/106762848 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580ad640 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd60ccb640 1 -- 172.21.15.133:0/106762848 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd580ad7a0 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.502+0000 7fdd60ccb640 1 -- 172.21.15.133:0/106762848 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580adb30 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- 172.21.15.133:0/106762848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c0a48d0 msgr2=0x7fdd5c09e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 --2- 172.21.15.133:0/106762848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c0a48d0 0x7fdd5c09e5b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c141bc0 tx=0x7fdd5801c3f0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- 172.21.15.133:0/106762848 shutdown_connections 2024-09-16T15:07:22.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 --2- 172.21.15.133:0/106762848 >> v2:172.21.15.191:3300/0 conn(0x7fdd5c13e530 0x7fdd5c14bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 --2- 172.21.15.133:0/106762848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c0a48d0 0x7fdd5c09e5b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- 172.21.15.133:0/106762848 >> 172.21.15.133:0/106762848 conn(0x7fdd5c066300 msgr2=0x7fdd5c0a3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:22.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- 172.21.15.133:0/106762848 shutdown_connections 2024-09-16T15:07:22.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- 172.21.15.133:0/106762848 wait complete. 2024-09-16T15:07:22.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 Processor -- start 2024-09-16T15:07:22.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- start start 2024-09-16T15:07:22.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5c09ff90 0x7fdd5c143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5c0a48d0 0x7fdd5c13da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c13e530 0x7fdd5c1dd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c13dfa0 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c136f60 con 0x7fdd5c09ff90 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c144a20 con 0x7fdd5c13e530 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd614cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5c0a48d0 0x7fdd5c13da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd624ce640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c13e530 0x7fdd5c1dd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd614cc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5c0a48d0 0x7fdd5c13da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51602/0 (socket says 172.21.15.133:51602) 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd614cc640 1 -- 172.21.15.133:0/775352106 learned_addr learned my addr 172.21.15.133:0/775352106 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd61ccd640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5c09ff90 0x7fdd5c143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd614cc640 1 -- 172.21.15.133:0/775352106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c13e530 msgr2=0x7fdd5c1dd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd614cc640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c13e530 0x7fdd5c1dd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd614cc640 1 -- 172.21.15.133:0/775352106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5c09ff90 msgr2=0x7fdd5c143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd614cc640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5c09ff90 0x7fdd5c143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd614cc640 1 -- 172.21.15.133:0/775352106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58009420 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd624ce640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c13e530 0x7fdd5c1dd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:22.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd614cc640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5c0a48d0 0x7fdd5c13da60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0011f0 tx=0x7fdd4c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:22.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd527fc640 1 -- 172.21.15.133:0/775352106 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0074e0 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd527fc640 1 -- 172.21.15.133:0/775352106 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c007640 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- 172.21.15.133:0/775352106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c144ce0 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd527fc640 1 -- 172.21.15.133:0/775352106 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004a10 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- 172.21.15.133:0/775352106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c143d30 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.506+0000 7fdd63f31640 1 -- 172.21.15.133:0/775352106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c09e5b0 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.510+0000 7fdd527fc640 1 -- 172.21.15.133:0/775352106 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fdd4c01c3b0 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.510+0000 7fdd527fc640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:22.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.510+0000 7fdd527fc640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:22.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.510+0000 7fdd527fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:22.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.510+0000 7fdd527fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:22.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.510+0000 7fdd527fc640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd40097870 0x7fdd40099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.510+0000 7fdd527fc640 1 -- 172.21.15.133:0/775352106 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 4362+0+0 (secure 0 0 0) 0x7fdd4c0bf4f0 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.510+0000 7fdd61ccd640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd40097870 0x7fdd40099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.510+0000 7fdd61ccd640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd40097870 0x7fdd40099d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdd580af000 tx=0x7fdd58008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:22.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.514+0000 7fdd527fc640 1 -- 172.21.15.133:0/775352106 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd4c08cc00 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.662+0000 7fdd63f31640 1 -- 172.21.15.133:0/775352106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd5c0a2360 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.662+0000 7fdd527fc640 1 -- 172.21.15.133:0/775352106 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13454 (secure 0 0 0) 0x7fdd4c018020 con 0x7fdd5c0a48d0 2024-09-16T15:07:22.668 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:22.668 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":20,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:22.047703+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:07:20.523939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-16T15:07:21.043356+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:22.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 10 mgrc shutdown 2024-09-16T15:07:22.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 -- 172.21.15.133:0/775352106 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd40097870 msgr2=0x7fdd40099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd40097870 0x7fdd40099d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdd580af000 tx=0x7fdd58008040 comp rx=0 tx=0).stop 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 -- 172.21.15.133:0/775352106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5c0a48d0 msgr2=0x7fdd5c13da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5c0a48d0 0x7fdd5c13da60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0011f0 tx=0x7fdd4c00c9b0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 -- 172.21.15.133:0/775352106 shutdown_connections 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd5c13e530 0x7fdd5c1dd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd40097870 0x7fdd40099d30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd5c0a48d0 0x7fdd5c13da60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 --2- 172.21.15.133:0/775352106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5c09ff90 0x7fdd5c143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 -- 172.21.15.133:0/775352106 >> 172.21.15.133:0/775352106 conn(0x7fdd5c066300 msgr2=0x7fdd5c0a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 -- 172.21.15.133:0/775352106 shutdown_connections 2024-09-16T15:07:22.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.666+0000 7fdd63f31640 1 -- 172.21.15.133:0/775352106 wait complete. 2024-09-16T15:07:22.685 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:07:22.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0c012ff640 1 Processor -- start 2024-09-16T15:07:22.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0c012ff640 1 -- start start 2024-09-16T15:07:22.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0c012ff640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0bfc06d180 0x7f0bfc0a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0c012ff640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bfc0a5e80 con 0x7f0bfc146bd0 2024-09-16T15:07:22.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0c012ff640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bfc09f7b0 con 0x7f0bfc0a5aa0 2024-09-16T15:07:22.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0c012ff640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bfc0a8470 con 0x7f0bfc06d180 2024-09-16T15:07:22.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfb7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0bfc146bd0 0x7f0bfc14df00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52514/0 (socket says 172.21.15.133:52514) 2024-09-16T15:07:22.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfb7fe640 1 -- 172.21.15.133:0/1384042073 learned_addr learned my addr 172.21.15.133:0/1384042073 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:22.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279281975 0 0) 0x7f0bfc0a5e80 con 0x7f0bfc146bd0 2024-09-16T15:07:22.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfa7fc640 1 --2- 172.21.15.133:0/1384042073 >> v2:172.21.15.191:3300/0 conn(0x7f0bfc06d180 0x7f0bfc0a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0be8003530 con 0x7f0bfc146bd0 2024-09-16T15:07:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2131633426 0 0) 0x7f0be8003530 con 0x7f0bfc146bd0 2024-09-16T15:07:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 >> v2:172.21.15.191:3300/0 conn(0x7f0bfc06d180 msgr2=0x7f0bfc0a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 --2- 172.21.15.133:0/1384042073 >> v2:172.21.15.191:3300/0 conn(0x7f0bfc06d180 0x7f0bfc0a3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 >> v1:172.21.15.191:6789/0 conn(0x7f0bfc0a5aa0 legacy=0x7f0bfc09f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc147320 con 0x7f0bfc146bd0 2024-09-16T15:07:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f0bf4001680 con 0x7f0bfc146bd0 2024-09-16T15:07:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 >> v1:172.21.15.133:6789/0 conn(0x7f0bfc146bd0 legacy=0x7f0bfc14df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0be8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0be80047d0 0x7f0be8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be8005190 0x7f0be800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0a8470 con 0x7f0be80047d0 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be800c0e0 con 0x7f0be8005190 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be800c2a0 con 0x7f0be8003980 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfa7fc640 1 --2- 172.21.15.133:0/1384042073 >> v2:172.21.15.191:3300/0 conn(0x7f0bfc06d180 0x7f0bfc0a3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfaffd640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0be8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfa7fc640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0be80047d0 0x7f0be8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfb7fe640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be8005190 0x7f0be800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfaffd640 1 -- 172.21.15.133:0/1384042073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be8005190 msgr2=0x7f0be800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfaffd640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0be8005190 0x7f0be800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfaffd640 1 -- 172.21.15.133:0/1384042073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0be80047d0 msgr2=0x7f0be8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfaffd640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0be80047d0 0x7f0be8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfaffd640 1 -- 172.21.15.133:0/1384042073 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be8003530 con 0x7f0be8003980 2024-09-16T15:07:22.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.802+0000 7f0bfb7fe640 1 -- 172.21.15.133:0/1384042073 reap_dead start 2024-09-16T15:07:22.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0bfaffd640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0be8003da0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0bec001e70 tx=0x7f0bec01c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cb6abfd46d2dc12 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:22.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec003b80 con 0x7f0be8003980 2024-09-16T15:07:22.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec003ce0 con 0x7f0be8003980 2024-09-16T15:07:22.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0bf9ffb640 1 -- 172.21.15.133:0/1384042073 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0052c0 con 0x7f0be8003980 2024-09-16T15:07:22.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 -- 172.21.15.133:0/1384042073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 msgr2=0x7f0be8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0be8003da0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0bec001e70 tx=0x7f0bec01c740 comp rx=0 tx=0).stop 2024-09-16T15:07:22.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 -- 172.21.15.133:0/1384042073 shutdown_connections 2024-09-16T15:07:22.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 --2- 172.21.15.133:0/1384042073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0be8003da0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 -- 172.21.15.133:0/1384042073 >> 172.21.15.133:0/1384042073 conn(0x7f0bfc066440 msgr2=0x7f0bfc0a5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:22.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 -- 172.21.15.133:0/1384042073 shutdown_connections 2024-09-16T15:07:22.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 -- 172.21.15.133:0/1384042073 wait complete. 2024-09-16T15:07:22.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 Processor -- start 2024-09-16T15:07:22.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 -- start start 2024-09-16T15:07:22.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0bfc071270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bfc0b4850 0x7f0bfc0a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bfc0a09f0 0x7f0bfc0a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0a28e0 con 0x7f0bfc0a09f0 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0b55d0 con 0x7f0bfc0b4850 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0c012ff640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc09e690 con 0x7f0be8003980 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0bfb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bfc0a09f0 0x7f0bfc0a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0bfaffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0bfc071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0bfaffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0bfc071270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:34120/0 (socket says 172.21.15.133:34120) 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0bfaffd640 1 -- 172.21.15.133:0/2601481983 learned_addr learned my addr 172.21.15.133:0/2601481983 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.806+0000 7f0bfa7fc640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bfc0b4850 0x7f0bfc0a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bfb7fe640 1 -- 172.21.15.133:0/2601481983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 msgr2=0x7f0bfc071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bfb7fe640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0bfc071270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bfb7fe640 1 -- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bfc0b4850 msgr2=0x7f0bfc0a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bfb7fe640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bfc0b4850 0x7f0bfc0a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bfb7fe640 1 -- 172.21.15.133:0/2601481983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc0b4d90 con 0x7f0bfc0a09f0 2024-09-16T15:07:22.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bfaffd640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0bfc071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bfa7fc640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bfc0b4850 0x7f0bfc0a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bfb7fe640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bfc0a09f0 0x7f0bfc0a0e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc06dea0 tx=0x7f0bf4011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bf37fe640 1 -- 172.21.15.133:0/2601481983 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400f8a0 con 0x7f0bfc0a09f0 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bf37fe640 1 -- 172.21.15.133:0/2601481983 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf400fa20 con 0x7f0bfc0a09f0 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0c012ff640 1 -- 172.21.15.133:0/2601481983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bfc0b5840 con 0x7f0bfc0a09f0 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bf37fe640 1 -- 172.21.15.133:0/2601481983 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400fdd0 con 0x7f0bfc0a09f0 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0c012ff640 1 -- 172.21.15.133:0/2601481983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc0b3b00 con 0x7f0bfc0a09f0 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0c012ff640 1 -- 172.21.15.133:0/2601481983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bfc0b4390 con 0x7f0bfc0a09f0 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bf37fe640 1 -- 172.21.15.133:0/2601481983 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f0bf401f460 con 0x7f0bfc0a09f0 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bf37fe640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:22.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bf37fe640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:22.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bf37fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:22.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bf37fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:22.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bf37fe640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0bdc097870 0x7f0bdc099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:22.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bf37fe640 1 -- 172.21.15.133:0/2601481983 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f0bf40c3640 con 0x7f0bfc0a09f0 2024-09-16T15:07:22.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bfaffd640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0bdc097870 0x7f0bdc099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:22.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.810+0000 7f0bfaffd640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0bdc097870 0x7f0bdc099d30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0bec003fd0 tx=0x7f0bec001520 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:22.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:22.814+0000 7f0bf37fe640 1 -- 172.21.15.133:0/2601481983 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0bf4090de0 con 0x7f0bfc0a09f0 2024-09-16T15:07:23.011 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.006+0000 7f0c012ff640 1 -- 172.21.15.133:0/2601481983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0bfc0a33d0 con 0x7f0bfc0a09f0 2024-09-16T15:07:23.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.718+0000 7f0bf37fe640 1 -- 172.21.15.133:0/2601481983 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0bfc0a33d0 con 0x7f0bfc0a09f0 2024-09-16T15:07:23.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 10 mgrc shutdown 2024-09-16T15:07:23.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 -- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0bdc097870 msgr2=0x7f0bdc099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:23.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0bdc097870 0x7f0bdc099d30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0bec003fd0 tx=0x7f0bec001520 comp rx=0 tx=0).stop 2024-09-16T15:07:23.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 -- 172.21.15.133:0/2601481983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bfc0a09f0 msgr2=0x7f0bfc0a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:23.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bfc0a09f0 0x7f0bfc0a0e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc06dea0 tx=0x7f0bf4011c20 comp rx=0 tx=0).stop 2024-09-16T15:07:23.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 -- 172.21.15.133:0/2601481983 shutdown_connections 2024-09-16T15:07:23.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0bdc097870 0x7f0bdc099d30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:23.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bfc0a09f0 0x7f0bfc0a0e50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:23.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bfc0b4850 0x7f0bfc0a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:23.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 --2- 172.21.15.133:0/2601481983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0be8003980 0x7f0bfc071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:23.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 -- 172.21.15.133:0/2601481983 >> 172.21.15.133:0/2601481983 conn(0x7f0bfc066440 msgr2=0x7f0bfc06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:23.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 -- 172.21.15.133:0/2601481983 shutdown_connections 2024-09-16T15:07:23.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.722+0000 7f0c012ff640 1 -- 172.21.15.133:0/2601481983 wait complete. 2024-09-16T15:07:23.749 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:07:23.750 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:07:23.860 INFO:tasks.ceph.mds.b.smithi191.stderr:2024-09-16T15:07:23.856+0000 7f4c13d01640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:07:23.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.870+0000 7fdfa94ba640 1 Processor -- start 2024-09-16T15:07:23.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.870+0000 7fdfa94ba640 1 -- start start 2024-09-16T15:07:23.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdfa40a2cf0 0x7fdfa40a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa40a1010 0x7fdfa414bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa406a8f0 con 0x7fdfa409e490 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa40a5bd0 con 0x7fdfa40a2cf0 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa40a5910 con 0x7fdfa40a1010 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa27fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdfa409e490 0x7fdfa406afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52520/0 (socket says 172.21.15.133:52520) 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa37fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa40a1010 0x7fdfa414bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa27fc640 1 -- 172.21.15.133:0/2551454252 learned_addr learned my addr 172.21.15.133:0/2551454252 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa2ffd640 1 --2- 172.21.15.133:0/2551454252 >> v2:172.21.15.191:3300/0 conn(0x7fdfa40a2cf0 0x7fdfa40a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa37fe640 1 -- 172.21.15.133:0/2551454252 >> v2:172.21.15.191:3300/0 conn(0x7fdfa40a2cf0 msgr2=0x7fdfa40a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa37fe640 1 --2- 172.21.15.133:0/2551454252 >> v2:172.21.15.191:3300/0 conn(0x7fdfa40a2cf0 0x7fdfa40a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa37fe640 1 -- 172.21.15.133:0/2551454252 >> v1:172.21.15.133:6789/0 conn(0x7fdfa409e490 legacy=0x7fdfa406afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa1ffb640 1 -- 172.21.15.133:0/2551454252 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946645125 0 0) 0x7fdfa406a8f0 con 0x7fdfa409e490 2024-09-16T15:07:23.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa37fe640 1 -- 172.21.15.133:0/2551454252 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa40a4cf0 con 0x7fdfa40a1010 2024-09-16T15:07:23.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa37fe640 1 --2- 172.21.15.133:0/2551454252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa40a1010 0x7fdfa414bd90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009d90 tx=0x7fdf9c01c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c521e019517bf39 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:23.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa1ffb640 1 -- 172.21.15.133:0/2551454252 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c0a58f0 con 0x7fdfa40a1010 2024-09-16T15:07:23.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa1ffb640 1 -- 172.21.15.133:0/2551454252 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c0a5a50 con 0x7fdfa40a1010 2024-09-16T15:07:23.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa1ffb640 1 -- 172.21.15.133:0/2551454252 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c0a5de0 con 0x7fdfa40a1010 2024-09-16T15:07:23.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 -- 172.21.15.133:0/2551454252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa40a1010 msgr2=0x7fdfa414bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:23.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 --2- 172.21.15.133:0/2551454252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa40a1010 0x7fdfa414bd90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009d90 tx=0x7fdf9c01c830 comp rx=0 tx=0).stop 2024-09-16T15:07:23.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 -- 172.21.15.133:0/2551454252 shutdown_connections 2024-09-16T15:07:23.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 --2- 172.21.15.133:0/2551454252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa40a1010 0x7fdfa414bd90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:23.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 --2- 172.21.15.133:0/2551454252 >> v2:172.21.15.191:3300/0 conn(0x7fdfa40a2cf0 0x7fdfa40a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:23.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.874+0000 7fdfa94ba640 1 -- 172.21.15.133:0/2551454252 >> 172.21.15.133:0/2551454252 conn(0x7fdfa40a3fd0 msgr2=0x7fdfa40a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:23.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 -- 172.21.15.133:0/2551454252 shutdown_connections 2024-09-16T15:07:23.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 -- 172.21.15.133:0/2551454252 wait complete. 2024-09-16T15:07:23.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 Processor -- start 2024-09-16T15:07:23.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 -- start start 2024-09-16T15:07:23.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa409e490 0x7fdfa4139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:23.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfa40a1010 0x7fdfa40b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:23.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa40a2cf0 0x7fdfa4141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:23.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa406ede0 con 0x7fdfa40a1010 2024-09-16T15:07:23.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa413a5b0 con 0x7fdfa40a2cf0 2024-09-16T15:07:23.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa413a230 con 0x7fdfa409e490 2024-09-16T15:07:23.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa2ffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa409e490 0x7fdfa4139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:23.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfa40a1010 0x7fdfa40b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:23.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa2ffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa409e490 0x7fdfa4139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:34146/0 (socket says 172.21.15.133:34146) 2024-09-16T15:07:23.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa2ffd640 1 -- 172.21.15.133:0/3944259507 learned_addr learned my addr 172.21.15.133:0/3944259507 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:23.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa37fe640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa40a2cf0 0x7fdfa4141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:23.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa27fc640 1 -- 172.21.15.133:0/3944259507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa409e490 msgr2=0x7fdfa4139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:23.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa27fc640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa409e490 0x7fdfa4139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:23.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa27fc640 1 -- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa40a2cf0 msgr2=0x7fdfa4141340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:23.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa27fc640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa40a2cf0 0x7fdfa4141340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:23.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa27fc640 1 -- 172.21.15.133:0/3944259507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c009a70 con 0x7fdfa40a1010 2024-09-16T15:07:23.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa37fe640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa40a2cf0 0x7fdfa4141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:23.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa2ffd640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa409e490 0x7fdfa4139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:23.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa27fc640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfa40a1010 0x7fdfa40b3470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800c430 tx=0x7fdf9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:23.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdf8b7fe640 1 -- 172.21.15.133:0/3944259507 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf980074e0 con 0x7fdfa40a1010 2024-09-16T15:07:23.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdf8b7fe640 1 -- 172.21.15.133:0/3944259507 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf98007660 con 0x7fdfa40a1010 2024-09-16T15:07:23.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 -- 172.21.15.133:0/3944259507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa406b870 con 0x7fdfa40a1010 2024-09-16T15:07:23.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdf8b7fe640 1 -- 172.21.15.133:0/3944259507 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98004a30 con 0x7fdfa40a1010 2024-09-16T15:07:23.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 -- 172.21.15.133:0/3944259507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa406e9c0 con 0x7fdfa40a1010 2024-09-16T15:07:23.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.878+0000 7fdfa94ba640 1 -- 172.21.15.133:0/3944259507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa406ef20 con 0x7fdfa40a1010 2024-09-16T15:07:23.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.882+0000 7fdf8b7fe640 1 -- 172.21.15.133:0/3944259507 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fdf98004b90 con 0x7fdfa40a1010 2024-09-16T15:07:23.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.882+0000 7fdf8b7fe640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:23.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.882+0000 7fdf8b7fe640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:23.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.882+0000 7fdf8b7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:23.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.882+0000 7fdf8b7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:23.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.882+0000 7fdf8b7fe640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdf900977c0 0x7fdf90099c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:23.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.882+0000 7fdf8b7fe640 1 -- 172.21.15.133:0/3944259507 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4362+0+0 (secure 0 0 0) 0x7fdf980c00a0 con 0x7fdfa40a1010 2024-09-16T15:07:23.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.882+0000 7fdfa2ffd640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdf900977c0 0x7fdf90099c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:23.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.882+0000 7fdfa2ffd640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdf900977c0 0x7fdf90099c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf94007e60 tx=0x7fdf94008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:23.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:23.886+0000 7fdf8b7fe640 1 -- 172.21.15.133:0/3944259507 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf9808d7b0 con 0x7fdfa40a1010 2024-09-16T15:07:24.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.066+0000 7fdfa94ba640 1 -- 172.21.15.133:0/3944259507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdfa406ef20 con 0x7fdfa40a1010 2024-09-16T15:07:24.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.726+0000 7fdf8b7fe640 1 -- 172.21.15.133:0/3944259507 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdfa406ef20 con 0x7fdfa40a1010 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 10 mgrc shutdown 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 -- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdf900977c0 msgr2=0x7fdf90099c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdf900977c0 0x7fdf90099c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf94007e60 tx=0x7fdf94008040 comp rx=0 tx=0).stop 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 -- 172.21.15.133:0/3944259507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfa40a1010 msgr2=0x7fdfa40b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfa40a1010 0x7fdfa40b3470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800c430 tx=0x7fdf9800c8f0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 -- 172.21.15.133:0/3944259507 shutdown_connections 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfa40a2cf0 0x7fdfa4141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdf900977c0 0x7fdf90099c80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfa40a1010 0x7fdfa40b3470 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 --2- 172.21.15.133:0/3944259507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfa409e490 0x7fdfa4139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 -- 172.21.15.133:0/3944259507 >> 172.21.15.133:0/3944259507 conn(0x7fdfa40a3fd0 msgr2=0x7fdfa40b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 -- 172.21.15.133:0/3944259507 shutdown_connections 2024-09-16T15:07:24.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.730+0000 7fdfa94ba640 1 -- 172.21.15.133:0/3944259507 wait complete. 2024-09-16T15:07:24.756 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:07:24.756 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:07:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.874+0000 7f1987faf640 1 Processor -- start 2024-09-16T15:07:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.874+0000 7f1987faf640 1 -- start start 2024-09-16T15:07:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.874+0000 7f1987faf640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f19800a6210 0x7f198009edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.874+0000 7f1987faf640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f19800a2fe0 0x7f1980140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.874+0000 7f1987faf640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198009f310 con 0x7f1980141110 2024-09-16T15:07:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.874+0000 7f1987faf640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19800a3590 con 0x7f19800a6210 2024-09-16T15:07:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.874+0000 7f1987faf640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19800a3850 con 0x7f19800a2fe0 2024-09-16T15:07:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f198654c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1980141110 0x7f1980143970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52534/0 (socket says 172.21.15.133:52534) 2024-09-16T15:07:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f198654c640 1 -- 172.21.15.133:0/830497385 learned_addr learned my addr 172.21.15.133:0/830497385 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1985d4b640 1 --2- 172.21.15.133:0/830497385 >> v2:172.21.15.133:3300/0 conn(0x7f19800a6210 0x7f198009edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f198554a640 1 --2- 172.21.15.133:0/830497385 >> v2:172.21.15.191:3300/0 conn(0x7f19800a2fe0 0x7f1980140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1984d49640 1 -- 172.21.15.133:0/830497385 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899824827 0 0) 0x7f1978001120 con 0x7f1980141110 2024-09-16T15:07:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1984d49640 1 -- 172.21.15.133:0/830497385 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f196c003530 con 0x7f1980141110 2024-09-16T15:07:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1985d4b640 1 -- 172.21.15.133:0/830497385 >> v2:172.21.15.191:3300/0 conn(0x7f19800a2fe0 msgr2=0x7f1980140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1985d4b640 1 --2- 172.21.15.133:0/830497385 >> v2:172.21.15.191:3300/0 conn(0x7f19800a2fe0 0x7f1980140bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1985d4b640 1 -- 172.21.15.133:0/830497385 >> v1:172.21.15.133:6789/0 conn(0x7f1980141110 legacy=0x7f1980143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1985d4b640 1 -- 172.21.15.133:0/830497385 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19801415c0 con 0x7f19800a6210 2024-09-16T15:07:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1985d4b640 1 --2- 172.21.15.133:0/830497385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19800a6210 0x7f198009edd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f19800a3fa0 tx=0x7f197401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a8b7afa6c54090 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1984d49640 1 -- 172.21.15.133:0/830497385 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740ab600 con 0x7f19800a6210 2024-09-16T15:07:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1984d49640 1 -- 172.21.15.133:0/830497385 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19740a7050 con 0x7f19800a6210 2024-09-16T15:07:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1984d49640 1 -- 172.21.15.133:0/830497385 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19740ab980 con 0x7f19800a6210 2024-09-16T15:07:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1987faf640 1 -- 172.21.15.133:0/830497385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19800a6210 msgr2=0x7f198009edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1987faf640 1 --2- 172.21.15.133:0/830497385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19800a6210 0x7f198009edd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f19800a3fa0 tx=0x7f197401c3d0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1987faf640 1 -- 172.21.15.133:0/830497385 shutdown_connections 2024-09-16T15:07:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1987faf640 1 --2- 172.21.15.133:0/830497385 >> v2:172.21.15.191:3300/0 conn(0x7f19800a2fe0 0x7f1980140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1987faf640 1 --2- 172.21.15.133:0/830497385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19800a6210 0x7f198009edd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1987faf640 1 -- 172.21.15.133:0/830497385 >> 172.21.15.133:0/830497385 conn(0x7f19800a3fd0 msgr2=0x7f19800a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1987faf640 1 -- 172.21.15.133:0/830497385 shutdown_connections 2024-09-16T15:07:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.878+0000 7f1987faf640 1 -- 172.21.15.133:0/830497385 wait complete. 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 Processor -- start 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 -- start start 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19800a2fe0 0x7f1980071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19800a6210 0x7f19801495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1980141110 0x7f19801deac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980099a90 con 0x7f1980141110 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980149f50 con 0x7f19800a2fe0 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198014a210 con 0x7f19800a6210 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f198654c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1980141110 0x7f19801deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f198554a640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19800a6210 0x7f19801495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f198654c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1980141110 0x7f19801deac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51658/0 (socket says 172.21.15.133:51658) 2024-09-16T15:07:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f198654c640 1 -- 172.21.15.133:0/1854135263 learned_addr learned my addr 172.21.15.133:0/1854135263 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1985d4b640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19800a2fe0 0x7f1980071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f198654c640 1 -- 172.21.15.133:0/1854135263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19800a6210 msgr2=0x7f19801495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f198654c640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19800a6210 0x7f19801495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f198654c640 1 -- 172.21.15.133:0/1854135263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19800a2fe0 msgr2=0x7f1980071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f198654c640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19800a2fe0 0x7f1980071190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f198654c640 1 -- 172.21.15.133:0/1854135263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1974009420 con 0x7f1980141110 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f198654c640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1980141110 0x7f19801deac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f197800c400 tx=0x7f197800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f19727fc640 1 -- 172.21.15.133:0/1854135263 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978005cf0 con 0x7f1980141110 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f19727fc640 1 -- 172.21.15.133:0/1854135263 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1978005e50 con 0x7f1980141110 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f19727fc640 1 -- 172.21.15.133:0/1854135263 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780072f0 con 0x7f1980141110 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 -- 172.21.15.133:0/1854135263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1980071e90 con 0x7f1980141110 2024-09-16T15:07:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 -- 172.21.15.133:0/1854135263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198009a080 con 0x7f1980141110 2024-09-16T15:07:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f1987faf640 1 -- 172.21.15.133:0/1854135263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1980140bd0 con 0x7f1980141110 2024-09-16T15:07:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f19727fc640 1 -- 172.21.15.133:0/1854135263 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f1978007450 con 0x7f1980141110 2024-09-16T15:07:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.882+0000 7f19727fc640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.886+0000 7f19727fc640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.886+0000 7f19727fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.886+0000 7f19727fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.886+0000 7f19727fc640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1964097870 0x7f1964099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.886+0000 7f19727fc640 1 -- 172.21.15.133:0/1854135263 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f19780bf1a0 con 0x7f1980141110 2024-09-16T15:07:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.886+0000 7f1985d4b640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1964097870 0x7f1964099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.886+0000 7f1985d4b640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1964097870 0x7f1964099d30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19740ad000 tx=0x7f1974008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:24.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:24.890+0000 7f19727fc640 1 -- 172.21.15.133:0/1854135263 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f19780903d0 con 0x7f1980141110 2024-09-16T15:07:25.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.082+0000 7f1987faf640 1 -- 172.21.15.133:0/1854135263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f198009f310 con 0x7f1980141110 2024-09-16T15:07:25.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.082+0000 7f19727fc640 1 -- 172.21.15.133:0/1854135263 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3978 (secure 0 0 0) 0x7f198009f310 con 0x7f1980141110 2024-09-16T15:07:25.087 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:25.088 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6811/3104644688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3104644688},{"type":"v1","addr":"172.21.15.133:6811","nonce":3104644688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6809/147821810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":147821810},{"type":"v1","addr":"172.21.15.133:6809","nonce":147821810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:07:22.048013+0000","modified":"2024-09-16T15:07:24.720576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6827/2751563681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2751563681},{"type":"v1","addr":"172.21.15.191:6827","nonce":2751563681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-16T15:07:25.088 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-09-16T15:07:25.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 10 mgrc shutdown 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 -- 172.21.15.133:0/1854135263 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1964097870 msgr2=0x7f1964099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1964097870 0x7f1964099d30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19740ad000 tx=0x7f1974008040 comp rx=0 tx=0).stop 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 -- 172.21.15.133:0/1854135263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1980141110 msgr2=0x7f19801deac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1980141110 0x7f19801deac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f197800c400 tx=0x7f197800c8c0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 -- 172.21.15.133:0/1854135263 shutdown_connections 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1964097870 0x7f1964099d30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1980141110 0x7f19801deac0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19800a6210 0x7f19801495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 --2- 172.21.15.133:0/1854135263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19800a2fe0 0x7f1980071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 -- 172.21.15.133:0/1854135263 >> 172.21.15.133:0/1854135263 conn(0x7f19800a3fd0 msgr2=0x7f19800a0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 -- 172.21.15.133:0/1854135263 shutdown_connections 2024-09-16T15:07:25.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.086+0000 7f1987faf640 1 -- 172.21.15.133:0/1854135263 wait complete. 2024-09-16T15:07:25.106 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:25.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd1777c640 1 Processor -- start 2024-09-16T15:07:25.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd1777c640 1 -- start start 2024-09-16T15:07:25.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd1777c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd1006ca20 0x7fcd100a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd1777c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1009f2a0 0x7fcd100b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd1777c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1009f7f0 con 0x7fcd100a38d0 2024-09-16T15:07:25.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd1777c640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1009fa20 con 0x7fcd1006ca20 2024-09-16T15:07:25.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd1777c640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd100a3cb0 con 0x7fcd1009f2a0 2024-09-16T15:07:25.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd15d19640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1009f2a0 0x7fcd100b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd15d19640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1009f2a0 0x7fcd100b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:34170/0 (socket says 172.21.15.133:34170) 2024-09-16T15:07:25.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd15d19640 1 -- 172.21.15.133:0/359980168 learned_addr learned my addr 172.21.15.133:0/359980168 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:25.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd14d17640 1 --2- 172.21.15.133:0/359980168 >> v2:172.21.15.191:3300/0 conn(0x7fcd1006ca20 0x7fcd100a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd15d19640 1 -- 172.21.15.133:0/359980168 >> v2:172.21.15.191:3300/0 conn(0x7fcd1006ca20 msgr2=0x7fcd100a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:25.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd15d19640 1 --2- 172.21.15.133:0/359980168 >> v2:172.21.15.191:3300/0 conn(0x7fcd1006ca20 0x7fcd100a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd15d19640 1 -- 172.21.15.133:0/359980168 >> v1:172.21.15.133:6789/0 conn(0x7fcd100a38d0 legacy=0x7fcd10141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.218+0000 7fcd15d19640 1 -- 172.21.15.133:0/359980168 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100a3df0 con 0x7fcd1009f2a0 2024-09-16T15:07:25.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd15d19640 1 --2- 172.21.15.133:0/359980168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1009f2a0 0x7fcd100b5ea0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009870 tx=0x7fcd0001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efd9f17f5b97d585 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:25.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd0ffff640 1 -- 172.21.15.133:0/359980168 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000ac620 con 0x7fcd1009f2a0 2024-09-16T15:07:25.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd0ffff640 1 -- 172.21.15.133:0/359980168 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd000ac780 con 0x7fcd1009f2a0 2024-09-16T15:07:25.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd0ffff640 1 -- 172.21.15.133:0/359980168 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000acb10 con 0x7fcd1009f2a0 2024-09-16T15:07:25.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- 172.21.15.133:0/359980168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1009f2a0 msgr2=0x7fcd100b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 --2- 172.21.15.133:0/359980168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1009f2a0 0x7fcd100b5ea0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009870 tx=0x7fcd0001c4b0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- 172.21.15.133:0/359980168 shutdown_connections 2024-09-16T15:07:25.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 --2- 172.21.15.133:0/359980168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1009f2a0 0x7fcd100b5ea0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 --2- 172.21.15.133:0/359980168 >> v2:172.21.15.191:3300/0 conn(0x7fcd1006ca20 0x7fcd100a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- 172.21.15.133:0/359980168 >> 172.21.15.133:0/359980168 conn(0x7fcd100a88e0 msgr2=0x7fcd100a8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:25.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- 172.21.15.133:0/359980168 shutdown_connections 2024-09-16T15:07:25.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- 172.21.15.133:0/359980168 wait complete. 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 Processor -- start 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- start start 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1006ca20 0x7fcd101453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1009f2a0 0x7fcd1013e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd100a38d0 0x7fcd100b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10144e00 con 0x7fcd1009f2a0 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10136f10 con 0x7fcd100a38d0 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10071c20 con 0x7fcd1006ca20 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd14d17640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1009f2a0 0x7fcd1013e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd15518640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1006ca20 0x7fcd101453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd14d17640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1009f2a0 0x7fcd1013e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51666/0 (socket says 172.21.15.133:51666) 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd14d17640 1 -- 172.21.15.133:0/2677885655 learned_addr learned my addr 172.21.15.133:0/2677885655 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd15518640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1006ca20 0x7fcd101453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:34172/0 (socket says 172.21.15.133:34172) 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd15d19640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd100a38d0 0x7fcd100b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd15518640 1 -- 172.21.15.133:0/2677885655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd100a38d0 msgr2=0x7fcd100b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd15518640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd100a38d0 0x7fcd100b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd15518640 1 -- 172.21.15.133:0/2677885655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1009f2a0 msgr2=0x7fcd1013e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd15518640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1009f2a0 0x7fcd1013e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd15518640 1 -- 172.21.15.133:0/2677885655 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd000094d0 con 0x7fcd1006ca20 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd15518640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1006ca20 0x7fcd101453f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800c680 tx=0x7fcd0800cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd0dffb640 1 -- 172.21.15.133:0/2677885655 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08003b80 con 0x7fcd1006ca20 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd0dffb640 1 -- 172.21.15.133:0/2677885655 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08003ce0 con 0x7fcd1006ca20 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- 172.21.15.133:0/2677885655 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd10071f00 con 0x7fcd1006ca20 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd0dffb640 1 -- 172.21.15.133:0/2677885655 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08005310 con 0x7fcd1006ca20 2024-09-16T15:07:25.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.222+0000 7fcd1777c640 1 -- 172.21.15.133:0/2677885655 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd100b58c0 con 0x7fcd1006ca20 2024-09-16T15:07:25.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.226+0000 7fcd1777c640 1 -- 172.21.15.133:0/2677885655 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd100a4b50 con 0x7fcd1006ca20 2024-09-16T15:07:25.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.226+0000 7fcd0dffb640 1 -- 172.21.15.133:0/2677885655 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fcd08005470 con 0x7fcd1006ca20 2024-09-16T15:07:25.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.226+0000 7fcd0dffb640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:25.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.226+0000 7fcd0dffb640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:25.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.226+0000 7fcd0dffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:25.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.226+0000 7fcd0dffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:25.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.226+0000 7fcd0dffb640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fccfc097840 0x7fccfc099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.226+0000 7fcd0dffb640 1 -- 172.21.15.133:0/2677885655 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4362+0+0 (secure 0 0 0) 0x7fcd080c04a0 con 0x7fcd1006ca20 2024-09-16T15:07:25.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.226+0000 7fcd14d17640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fccfc097840 0x7fccfc099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.226+0000 7fcd14d17640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fccfc097840 0x7fccfc099d00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd10142770 tx=0x7fcd04008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:25.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.230+0000 7fcd0dffb640 1 -- 172.21.15.133:0/2677885655 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcd0808dba0 con 0x7fcd1006ca20 2024-09-16T15:07:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.382+0000 7fcd1777c640 1 -- 172.21.15.133:0/2677885655 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd100b4c50 con 0x7fcd1006ca20 2024-09-16T15:07:25.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd0dffb640 1 -- 172.21.15.133:0/2677885655 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+13441 (secure 0 0 0) 0x7fcd080911c0 con 0x7fcd1006ca20 2024-09-16T15:07:25.390 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:25.390 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:23.049976+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:07:20.523939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-16T15:07:21.043356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 10 mgrc shutdown 2024-09-16T15:07:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 -- 172.21.15.133:0/2677885655 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fccfc097840 msgr2=0x7fccfc099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fccfc097840 0x7fccfc099d00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd10142770 tx=0x7fcd04008040 comp rx=0 tx=0).stop 2024-09-16T15:07:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 -- 172.21.15.133:0/2677885655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1006ca20 msgr2=0x7fcd101453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1006ca20 0x7fcd101453f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800c680 tx=0x7fcd0800cb40 comp rx=0 tx=0).stop 2024-09-16T15:07:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 -- 172.21.15.133:0/2677885655 shutdown_connections 2024-09-16T15:07:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd100a38d0 0x7fcd100b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fccfc097840 0x7fccfc099d00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd1009f2a0 0x7fcd1013e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 --2- 172.21.15.133:0/2677885655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd1006ca20 0x7fcd101453f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 -- 172.21.15.133:0/2677885655 >> 172.21.15.133:0/2677885655 conn(0x7fcd100a88e0 msgr2=0x7fcd100a16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 -- 172.21.15.133:0/2677885655 shutdown_connections 2024-09-16T15:07:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.386+0000 7fcd1777c640 1 -- 172.21.15.133:0/2677885655 wait complete. 2024-09-16T15:07:25.407 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-16T15:07:25.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:07:25.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.506+0000 7f7f2e86d640 1 Processor -- start 2024-09-16T15:07:25.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.506+0000 7f7f2e86d640 1 -- start start 2024-09-16T15:07:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7f280a6210 0x7f7f2809edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2809f310 con 0x7f7f28141110 2024-09-16T15:07:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f280a3590 con 0x7f7f280a6210 2024-09-16T15:07:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f280a3850 con 0x7f7f280a2fe0 2024-09-16T15:07:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f277fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f27fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7f280a6210 0x7f7f2809edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2ce0a640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7f28141110 0x7f7f28143970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52548/0 (socket says 172.21.15.133:52548) 2024-09-16T15:07:25.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2ce0a640 1 -- 172.21.15.133:0/2377322042 learned_addr learned my addr 172.21.15.133:0/2377322042 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:25.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f277fe640 1 -- 172.21.15.133:0/2377322042 >> v2:172.21.15.133:3300/0 conn(0x7f7f280a6210 msgr2=0x7f7f2809edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:25.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f277fe640 1 --2- 172.21.15.133:0/2377322042 >> v2:172.21.15.133:3300/0 conn(0x7f7f280a6210 0x7f7f2809edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f277fe640 1 -- 172.21.15.133:0/2377322042 >> v1:172.21.15.133:6789/0 conn(0x7f7f28141110 legacy=0x7f7f28143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f26ffd640 1 -- 172.21.15.133:0/2377322042 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263047396 0 0) 0x7f7f200014a0 con 0x7f7f28141110 2024-09-16T15:07:25.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f277fe640 1 -- 172.21.15.133:0/2377322042 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f281415c0 con 0x7f7f280a2fe0 2024-09-16T15:07:25.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f27fff640 1 --2- 172.21.15.133:0/2377322042 >> v2:172.21.15.133:3300/0 conn(0x7f7f280a6210 0x7f7f2809edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:25.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f277fe640 1 --2- 172.21.15.133:0/2377322042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28140bd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7f180097f0 tx=0x7f7f1801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcead999493f41d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:25.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f26ffd640 1 -- 172.21.15.133:0/2377322042 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f180ab600 con 0x7f7f280a2fe0 2024-09-16T15:07:25.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f26ffd640 1 -- 172.21.15.133:0/2377322042 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f180a7050 con 0x7f7f280a2fe0 2024-09-16T15:07:25.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f26ffd640 1 -- 172.21.15.133:0/2377322042 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f180ab980 con 0x7f7f280a2fe0 2024-09-16T15:07:25.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 -- 172.21.15.133:0/2377322042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 msgr2=0x7f7f28140bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 --2- 172.21.15.133:0/2377322042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28140bd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7f180097f0 tx=0x7f7f1801c3f0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 -- 172.21.15.133:0/2377322042 shutdown_connections 2024-09-16T15:07:25.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 --2- 172.21.15.133:0/2377322042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28140bd0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 --2- 172.21.15.133:0/2377322042 >> v2:172.21.15.133:3300/0 conn(0x7f7f280a6210 0x7f7f2809edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 -- 172.21.15.133:0/2377322042 >> 172.21.15.133:0/2377322042 conn(0x7f7f280a3fd0 msgr2=0x7f7f280a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:25.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 -- 172.21.15.133:0/2377322042 shutdown_connections 2024-09-16T15:07:25.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.510+0000 7f7f2e86d640 1 -- 172.21.15.133:0/2377322042 wait complete. 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 Processor -- start 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 -- start start 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f280a6210 0x7f7f28145640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f28141110 0x7f7f2809c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f281dd210 con 0x7f7f280a6210 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f281dd4b0 con 0x7f7f28141110 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2809aec0 con 0x7f7f280a2fe0 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f27fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f277fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f280a6210 0x7f7f28145640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f27fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:34188/0 (socket says 172.21.15.133:34188) 2024-09-16T15:07:25.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f27fff640 1 -- 172.21.15.133:0/337359407 learned_addr learned my addr 172.21.15.133:0/337359407 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2ce0a640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f28141110 0x7f7f2809c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f27fff640 1 -- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f28141110 msgr2=0x7f7f2809c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f27fff640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f28141110 0x7f7f2809c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f27fff640 1 -- 172.21.15.133:0/337359407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f280a6210 msgr2=0x7f7f28145640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f27fff640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f280a6210 0x7f7f28145640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f27fff640 1 -- 172.21.15.133:0/337359407 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f10009420 con 0x7f7f280a2fe0 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f277fe640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f280a6210 0x7f7f28145640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2ce0a640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f28141110 0x7f7f2809c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f27fff640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28071590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000ebe0 tx=0x7f7f1000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f24ff9640 1 -- 172.21.15.133:0/337359407 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000eee0 con 0x7f7f280a2fe0 2024-09-16T15:07:25.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f24ff9640 1 -- 172.21.15.133:0/337359407 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1000cc30 con 0x7f7f280a2fe0 2024-09-16T15:07:25.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 -- 172.21.15.133:0/337359407 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f18009420 con 0x7f7f280a2fe0 2024-09-16T15:07:25.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f24ff9640 1 -- 172.21.15.133:0/337359407 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10013470 con 0x7f7f280a2fe0 2024-09-16T15:07:25.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 -- 172.21.15.133:0/337359407 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2809ab70 con 0x7f7f280a2fe0 2024-09-16T15:07:25.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.514+0000 7f7f2e86d640 1 -- 172.21.15.133:0/337359407 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f28140bd0 con 0x7f7f280a2fe0 2024-09-16T15:07:25.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.518+0000 7f7f24ff9640 1 -- 172.21.15.133:0/337359407 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f7f100135d0 con 0x7f7f280a2fe0 2024-09-16T15:07:25.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.518+0000 7f7f24ff9640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:25.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.518+0000 7f7f24ff9640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:25.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.518+0000 7f7f24ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:25.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.518+0000 7f7f24ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:25.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.518+0000 7f7f24ff9640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7f08097870 0x7f7f08099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.518+0000 7f7f24ff9640 1 -- 172.21.15.133:0/337359407 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f7f100c0780 con 0x7f7f280a2fe0 2024-09-16T15:07:25.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.518+0000 7f7f277fe640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7f08097870 0x7f7f08099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.518+0000 7f7f277fe640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7f08097870 0x7f7f08099d30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f180ad000 tx=0x7f7f180a6000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:25.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.522+0000 7f7f24ff9640 1 -- 172.21.15.133:0/337359407 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7f1008de90 con 0x7f7f280a2fe0 2024-09-16T15:07:25.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.666+0000 7f7f2e86d640 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2024-09-16T15:07:25.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.666+0000 7f7f2e86d640 1 -- 172.21.15.133:0/337359407 --> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f280b5540 con 0x7f7f08097870 2024-09-16T15:07:25.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.686+0000 7f7f24ff9640 1 -- 172.21.15.133:0/337359407 <== mgr.4102 v2:172.21.15.191:6824/3926506186 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271683 (secure 0 0 0) 0x7f7f280b5540 con 0x7f7f08097870 2024-09-16T15:07:25.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.686+0000 7f7f24ff9640 20 mgrc handle_command_reply tid 0 r 0 2024-09-16T15:07:25.693 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:25.697 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-16T15:07:25.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 10 mgrc shutdown 2024-09-16T15:07:25.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 -- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7f08097870 msgr2=0x7f7f08099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7f08097870 0x7f7f08099d30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f180ad000 tx=0x7f7f180a6000 comp rx=0 tx=0).stop 2024-09-16T15:07:25.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 -- 172.21.15.133:0/337359407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 msgr2=0x7f7f28071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28071590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000ebe0 tx=0x7f7f1000c520 comp rx=0 tx=0).stop 2024-09-16T15:07:25.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 -- 172.21.15.133:0/337359407 shutdown_connections 2024-09-16T15:07:25.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f28141110 0x7f7f2809c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7f08097870 0x7f7f08099d30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f280a6210 0x7f7f28145640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 --2- 172.21.15.133:0/337359407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f280a2fe0 0x7f7f28071590 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 -- 172.21.15.133:0/337359407 >> 172.21.15.133:0/337359407 conn(0x7f7f280a3fd0 msgr2=0x7f7f280a0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:25.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 -- 172.21.15.133:0/337359407 shutdown_connections 2024-09-16T15:07:25.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.694+0000 7f7f2e86d640 1 -- 172.21.15.133:0/337359407 wait complete. 2024-09-16T15:07:25.712 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-16T15:07:24.690268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592749,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":205,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":109988,"kb_used_data":1892,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377377372,"statfs":{"total":386547056640,"available":386434428928,"internally_reserved":0,"allocated":1937408,"data_stored":1423802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6374,"internal_metadata":110618394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020614"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.078209+0000","last_change":"2024-09-16T15:07:22.073492+0000","last_active":"2024-09-16T15:07:22.078209+0000","last_peered":"2024-09-16T15:07:22.078209+0000","last_clean":"2024-09-16T15:07:22.078209+0000","last_became_active":"2024-09-16T15:07:22.073186+0000","last_became_peered":"2024-09-16T15:07:22.073186+0000","last_unstale":"2024-09-16T15:07:22.078209+0000","last_undegraded":"2024-09-16T15:07:22.078209+0000","last_fullsized":"2024-09-16T15:07:22.078209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.091001+0000","last_change":"2024-09-16T15:07:22.089956+0000","last_active":"2024-09-16T15:07:22.091001+0000","last_peered":"2024-09-16T15:07:22.091001+0000","last_clean":"2024-09-16T15:07:22.091001+0000","last_became_active":"2024-09-16T15:07:22.089659+0000","last_became_peered":"2024-09-16T15:07:22.089659+0000","last_unstale":"2024-09-16T15:07:22.091001+0000","last_undegraded":"2024-09-16T15:07:22.091001+0000","last_fullsized":"2024-09-16T15:07:22.091001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:14.566411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.095057+0000","last_change":"2024-09-16T15:07:22.092432+0000","last_active":"2024-09-16T15:07:22.095057+0000","last_peered":"2024-09-16T15:07:22.095057+0000","last_clean":"2024-09-16T15:07:22.095057+0000","last_became_active":"2024-09-16T15:07:22.092188+0000","last_became_peered":"2024-09-16T15:07:22.092188+0000","last_unstale":"2024-09-16T15:07:22.095057+0000","last_undegraded":"2024-09-16T15:07:22.095057+0000","last_fullsized":"2024-09-16T15:07:22.095057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:25.997418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.098144+0000","last_change":"2024-09-16T15:07:22.095720+0000","last_active":"2024-09-16T15:07:22.098144+0000","last_peered":"2024-09-16T15:07:22.098144+0000","last_clean":"2024-09-16T15:07:22.098144+0000","last_became_active":"2024-09-16T15:07:22.093626+0000","last_became_peered":"2024-09-16T15:07:22.093626+0000","last_unstale":"2024-09-16T15:07:22.098144+0000","last_undegraded":"2024-09-16T15:07:22.098144+0000","last_fullsized":"2024-09-16T15:07:22.098144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:24.351732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082976+0000","last_change":"2024-09-16T15:07:22.077679+0000","last_active":"2024-09-16T15:07:22.082976+0000","last_peered":"2024-09-16T15:07:22.082976+0000","last_clean":"2024-09-16T15:07:22.082976+0000","last_became_active":"2024-09-16T15:07:22.077387+0000","last_became_peered":"2024-09-16T15:07:22.077387+0000","last_unstale":"2024-09-16T15:07:22.082976+0000","last_undegraded":"2024-09-16T15:07:22.082976+0000","last_fullsized":"2024-09-16T15:07:22.082976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:09.497997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082851+0000","last_change":"2024-09-16T15:07:22.077568+0000","last_active":"2024-09-16T15:07:22.082851+0000","last_peered":"2024-09-16T15:07:22.082851+0000","last_clean":"2024-09-16T15:07:22.082851+0000","last_became_active":"2024-09-16T15:07:22.076638+0000","last_became_peered":"2024-09-16T15:07:22.076638+0000","last_unstale":"2024-09-16T15:07:22.082851+0000","last_undegraded":"2024-09-16T15:07:22.082851+0000","last_fullsized":"2024-09-16T15:07:22.082851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:28.264456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.081204+0000","last_change":"2024-09-16T15:07:22.075898+0000","last_active":"2024-09-16T15:07:22.081204+0000","last_peered":"2024-09-16T15:07:22.081204+0000","last_clean":"2024-09-16T15:07:22.081204+0000","last_became_active":"2024-09-16T15:07:22.075448+0000","last_became_peered":"2024-09-16T15:07:22.075448+0000","last_unstale":"2024-09-16T15:07:22.081204+0000","last_undegraded":"2024-09-16T15:07:22.081204+0000","last_fullsized":"2024-09-16T15:07:22.081204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:03.093921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.101627+0000","last_change":"2024-09-16T15:07:22.096662+0000","last_active":"2024-09-16T15:07:22.101627+0000","last_peered":"2024-09-16T15:07:22.101627+0000","last_clean":"2024-09-16T15:07:22.101627+0000","last_became_active":"2024-09-16T15:07:22.095671+0000","last_became_peered":"2024-09-16T15:07:22.095671+0000","last_unstale":"2024-09-16T15:07:22.101627+0000","last_undegraded":"2024-09-16T15:07:22.101627+0000","last_fullsized":"2024-09-16T15:07:22.101627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:47.108958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093209+0000","last_change":"2024-09-16T15:07:22.092184+0000","last_active":"2024-09-16T15:07:22.093209+0000","last_peered":"2024-09-16T15:07:22.093209+0000","last_clean":"2024-09-16T15:07:22.093209+0000","last_became_active":"2024-09-16T15:07:22.091896+0000","last_became_peered":"2024-09-16T15:07:22.091896+0000","last_unstale":"2024-09-16T15:07:22.093209+0000","last_undegraded":"2024-09-16T15:07:22.093209+0000","last_fullsized":"2024-09-16T15:07:22.093209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.098099+0000","last_change":"2024-09-16T15:07:22.093487+0000","last_active":"2024-09-16T15:07:22.098099+0000","last_peered":"2024-09-16T15:07:22.098099+0000","last_clean":"2024-09-16T15:07:22.098099+0000","last_became_active":"2024-09-16T15:07:22.091134+0000","last_became_peered":"2024-09-16T15:07:22.091134+0000","last_unstale":"2024-09-16T15:07:22.098099+0000","last_undegraded":"2024-09-16T15:07:22.098099+0000","last_fullsized":"2024-09-16T15:07:22.098099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:55.259949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097499+0000","last_change":"2024-09-16T15:07:22.091123+0000","last_active":"2024-09-16T15:07:22.097499+0000","last_peered":"2024-09-16T15:07:22.097499+0000","last_clean":"2024-09-16T15:07:22.097499+0000","last_became_active":"2024-09-16T15:07:22.090822+0000","last_became_peered":"2024-09-16T15:07:22.090822+0000","last_unstale":"2024-09-16T15:07:22.097499+0000","last_undegraded":"2024-09-16T15:07:22.097499+0000","last_fullsized":"2024-09-16T15:07:22.097499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.081454+0000","last_change":"2024-09-16T15:07:22.075137+0000","last_active":"2024-09-16T15:07:22.081454+0000","last_peered":"2024-09-16T15:07:22.081454+0000","last_clean":"2024-09-16T15:07:22.081454+0000","last_became_active":"2024-09-16T15:07:22.074779+0000","last_became_peered":"2024-09-16T15:07:22.074779+0000","last_unstale":"2024-09-16T15:07:22.081454+0000","last_undegraded":"2024-09-16T15:07:22.081454+0000","last_fullsized":"2024-09-16T15:07:22.081454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:34.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.070550+0000","last_change":"2024-09-16T15:07:22.069681+0000","last_active":"2024-09-16T15:07:22.070550+0000","last_peered":"2024-09-16T15:07:22.070550+0000","last_clean":"2024-09-16T15:07:22.070550+0000","last_became_active":"2024-09-16T15:07:22.069331+0000","last_became_peered":"2024-09-16T15:07:22.069331+0000","last_unstale":"2024-09-16T15:07:22.070550+0000","last_undegraded":"2024-09-16T15:07:22.070550+0000","last_fullsized":"2024-09-16T15:07:22.070550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:27.703305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097717+0000","last_change":"2024-09-16T15:07:22.093463+0000","last_active":"2024-09-16T15:07:22.097717+0000","last_peered":"2024-09-16T15:07:22.097717+0000","last_clean":"2024-09-16T15:07:22.097717+0000","last_became_active":"2024-09-16T15:07:22.091154+0000","last_became_peered":"2024-09-16T15:07:22.091154+0000","last_unstale":"2024-09-16T15:07:22.097717+0000","last_undegraded":"2024-09-16T15:07:22.097717+0000","last_fullsized":"2024-09-16T15:07:22.097717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:24:09.086333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093577+0000","last_change":"2024-09-16T15:07:22.087589+0000","last_active":"2024-09-16T15:07:22.093577+0000","last_peered":"2024-09-16T15:07:22.093577+0000","last_clean":"2024-09-16T15:07:22.093577+0000","last_became_active":"2024-09-16T15:07:22.087167+0000","last_became_peered":"2024-09-16T15:07:22.087167+0000","last_unstale":"2024-09-16T15:07:22.093577+0000","last_undegraded":"2024-09-16T15:07:22.093577+0000","last_fullsized":"2024-09-16T15:07:22.093577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:26.349524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.081253+0000","last_change":"2024-09-16T15:07:22.076065+0000","last_active":"2024-09-16T15:07:22.081253+0000","last_peered":"2024-09-16T15:07:22.081253+0000","last_clean":"2024-09-16T15:07:22.081253+0000","last_became_active":"2024-09-16T15:07:22.075724+0000","last_became_peered":"2024-09-16T15:07:22.075724+0000","last_unstale":"2024-09-16T15:07:22.081253+0000","last_undegraded":"2024-09-16T15:07:22.081253+0000","last_fullsized":"2024-09-16T15:07:22.081253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:20:30.555559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.099304+0000","last_change":"2024-09-16T15:07:22.091007+0000","last_active":"2024-09-16T15:07:22.099304+0000","last_peered":"2024-09-16T15:07:22.099304+0000","last_clean":"2024-09-16T15:07:22.099304+0000","last_became_active":"2024-09-16T15:07:22.089216+0000","last_became_peered":"2024-09-16T15:07:22.089216+0000","last_unstale":"2024-09-16T15:07:22.099304+0000","last_undegraded":"2024-09-16T15:07:22.099304+0000","last_fullsized":"2024-09-16T15:07:22.099304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:43.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082877+0000","last_change":"2024-09-16T15:07:22.077822+0000","last_active":"2024-09-16T15:07:22.082877+0000","last_peered":"2024-09-16T15:07:22.082877+0000","last_clean":"2024-09-16T15:07:22.082877+0000","last_became_active":"2024-09-16T15:07:22.076545+0000","last_became_peered":"2024-09-16T15:07:22.076545+0000","last_unstale":"2024-09-16T15:07:22.082877+0000","last_undegraded":"2024-09-16T15:07:22.082877+0000","last_fullsized":"2024-09-16T15:07:22.082877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:03.892835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.075184+0000","last_change":"2024-09-16T15:07:22.074503+0000","last_active":"2024-09-16T15:07:22.075184+0000","last_peered":"2024-09-16T15:07:22.075184+0000","last_clean":"2024-09-16T15:07:22.075184+0000","last_became_active":"2024-09-16T15:07:22.074240+0000","last_became_peered":"2024-09-16T15:07:22.074240+0000","last_unstale":"2024-09-16T15:07:22.075184+0000","last_undegraded":"2024-09-16T15:07:22.075184+0000","last_fullsized":"2024-09-16T15:07:22.075184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:45.906789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097702+0000","last_change":"2024-09-16T15:07:22.090412+0000","last_active":"2024-09-16T15:07:22.097702+0000","last_peered":"2024-09-16T15:07:22.097702+0000","last_clean":"2024-09-16T15:07:22.097702+0000","last_became_active":"2024-09-16T15:07:22.089178+0000","last_became_peered":"2024-09-16T15:07:22.089178+0000","last_unstale":"2024-09-16T15:07:22.097702+0000","last_undegraded":"2024-09-16T15:07:22.097702+0000","last_fullsized":"2024-09-16T15:07:22.097702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:20.559503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.088950+0000","last_change":"2024-09-16T15:07:22.085622+0000","last_active":"2024-09-16T15:07:22.088950+0000","last_peered":"2024-09-16T15:07:22.088950+0000","last_clean":"2024-09-16T15:07:22.088950+0000","last_became_active":"2024-09-16T15:07:22.085321+0000","last_became_peered":"2024-09-16T15:07:22.085321+0000","last_unstale":"2024-09-16T15:07:22.088950+0000","last_undegraded":"2024-09-16T15:07:22.088950+0000","last_fullsized":"2024-09-16T15:07:22.088950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:25.939312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093696+0000","last_change":"2024-09-16T15:07:22.088691+0000","last_active":"2024-09-16T15:07:22.093696+0000","last_peered":"2024-09-16T15:07:22.093696+0000","last_clean":"2024-09-16T15:07:22.093696+0000","last_became_active":"2024-09-16T15:07:22.085505+0000","last_became_peered":"2024-09-16T15:07:22.085505+0000","last_unstale":"2024-09-16T15:07:22.093696+0000","last_undegraded":"2024-09-16T15:07:22.093696+0000","last_fullsized":"2024-09-16T15:07:22.093696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:20.387713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.095357+0000","last_change":"2024-09-16T15:07:22.093287+0000","last_active":"2024-09-16T15:07:22.095357+0000","last_peered":"2024-09-16T15:07:22.095357+0000","last_clean":"2024-09-16T15:07:22.095357+0000","last_became_active":"2024-09-16T15:07:22.092983+0000","last_became_peered":"2024-09-16T15:07:22.092983+0000","last_unstale":"2024-09-16T15:07:22.095357+0000","last_undegraded":"2024-09-16T15:07:22.095357+0000","last_fullsized":"2024-09-16T15:07:22.095357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:32.532671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.099462+0000","last_change":"2024-09-16T15:07:22.096232+0000","last_active":"2024-09-16T15:07:22.099462+0000","last_peered":"2024-09-16T15:07:22.099462+0000","last_clean":"2024-09-16T15:07:22.099462+0000","last_became_active":"2024-09-16T15:07:22.091234+0000","last_became_peered":"2024-09-16T15:07:22.091234+0000","last_unstale":"2024-09-16T15:07:22.099462+0000","last_undegraded":"2024-09-16T15:07:22.099462+0000","last_fullsized":"2024-09-16T15:07:22.099462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:23.826942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093633+0000","last_change":"2024-09-16T15:07:22.088597+0000","last_active":"2024-09-16T15:07:22.093633+0000","last_peered":"2024-09-16T15:07:22.093633+0000","last_clean":"2024-09-16T15:07:22.093633+0000","last_became_active":"2024-09-16T15:07:22.086951+0000","last_became_peered":"2024-09-16T15:07:22.086951+0000","last_unstale":"2024-09-16T15:07:22.093633+0000","last_undegraded":"2024-09-16T15:07:22.093633+0000","last_fullsized":"2024-09-16T15:07:22.093633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:57.389973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.070114+0000","last_change":"2024-09-16T15:07:22.067847+0000","last_active":"2024-09-16T15:07:22.070114+0000","last_peered":"2024-09-16T15:07:22.070114+0000","last_clean":"2024-09-16T15:07:22.070114+0000","last_became_active":"2024-09-16T15:07:22.067337+0000","last_became_peered":"2024-09-16T15:07:22.067337+0000","last_unstale":"2024-09-16T15:07:22.070114+0000","last_undegraded":"2024-09-16T15:07:22.070114+0000","last_fullsized":"2024-09-16T15:07:22.070114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:23.826942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.083117+0000","last_change":"2024-09-16T15:07:22.079269+0000","last_active":"2024-09-16T15:07:22.083117+0000","last_peered":"2024-09-16T15:07:22.083117+0000","last_clean":"2024-09-16T15:07:22.083117+0000","last_became_active":"2024-09-16T15:07:22.078377+0000","last_became_peered":"2024-09-16T15:07:22.078377+0000","last_unstale":"2024-09-16T15:07:22.083117+0000","last_undegraded":"2024-09-16T15:07:22.083117+0000","last_fullsized":"2024-09-16T15:07:22.083117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:51.357090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.069644+0000","last_change":"2024-09-16T15:07:22.067944+0000","last_active":"2024-09-16T15:07:22.069644+0000","last_peered":"2024-09-16T15:07:22.069644+0000","last_clean":"2024-09-16T15:07:22.069644+0000","last_became_active":"2024-09-16T15:07:22.067670+0000","last_became_peered":"2024-09-16T15:07:22.067670+0000","last_unstale":"2024-09-16T15:07:22.069644+0000","last_undegraded":"2024-09-16T15:07:22.069644+0000","last_fullsized":"2024-09-16T15:07:22.069644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:10.707281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.075117+0000","last_change":"2024-09-16T15:07:22.074211+0000","last_active":"2024-09-16T15:07:22.075117+0000","last_peered":"2024-09-16T15:07:22.075117+0000","last_clean":"2024-09-16T15:07:22.075117+0000","last_became_active":"2024-09-16T15:07:22.073954+0000","last_became_peered":"2024-09-16T15:07:22.073954+0000","last_unstale":"2024-09-16T15:07:22.075117+0000","last_undegraded":"2024-09-16T15:07:22.075117+0000","last_fullsized":"2024-09-16T15:07:22.075117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:51.922251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097535+0000","last_change":"2024-09-16T15:07:22.088989+0000","last_active":"2024-09-16T15:07:22.097535+0000","last_peered":"2024-09-16T15:07:22.097535+0000","last_clean":"2024-09-16T15:07:22.097535+0000","last_became_active":"2024-09-16T15:07:22.085895+0000","last_became_peered":"2024-09-16T15:07:22.085895+0000","last_unstale":"2024-09-16T15:07:22.097535+0000","last_undegraded":"2024-09-16T15:07:22.097535+0000","last_fullsized":"2024-09-16T15:07:22.097535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:58.765327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093222+0000","last_change":"2024-09-16T15:07:22.092307+0000","last_active":"2024-09-16T15:07:22.093222+0000","last_peered":"2024-09-16T15:07:22.093222+0000","last_clean":"2024-09-16T15:07:22.093222+0000","last_became_active":"2024-09-16T15:07:22.091941+0000","last_became_peered":"2024-09-16T15:07:22.091941+0000","last_unstale":"2024-09-16T15:07:22.093222+0000","last_undegraded":"2024-09-16T15:07:22.093222+0000","last_fullsized":"2024-09-16T15:07:22.093222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:14.566411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.099446+0000","last_change":"2024-09-16T15:07:22.096217+0000","last_active":"2024-09-16T15:07:22.099446+0000","last_peered":"2024-09-16T15:07:22.099446+0000","last_clean":"2024-09-16T15:07:22.099446+0000","last_became_active":"2024-09-16T15:07:22.092983+0000","last_became_peered":"2024-09-16T15:07:22.092983+0000","last_unstale":"2024-09-16T15:07:22.099446+0000","last_undegraded":"2024-09-16T15:07:22.099446+0000","last_fullsized":"2024-09-16T15:07:22.099446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:04.412982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.083172+0000","last_change":"2024-09-16T15:07:22.079554+0000","last_active":"2024-09-16T15:07:22.083172+0000","last_peered":"2024-09-16T15:07:22.083172+0000","last_clean":"2024-09-16T15:07:22.083172+0000","last_became_active":"2024-09-16T15:07:22.078495+0000","last_became_peered":"2024-09-16T15:07:22.078495+0000","last_unstale":"2024-09-16T15:07:22.083172+0000","last_undegraded":"2024-09-16T15:07:22.083172+0000","last_fullsized":"2024-09-16T15:07:22.083172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:25.939312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.077298+0000","last_change":"2024-09-16T15:07:21.742735+0000","last_active":"2024-09-16T15:07:22.077298+0000","last_peered":"2024-09-16T15:07:22.077298+0000","last_clean":"2024-09-16T15:07:22.077298+0000","last_became_active":"2024-09-16T15:07:21.742323+0000","last_became_peered":"2024-09-16T15:07:21.742323+0000","last_unstale":"2024-09-16T15:07:22.077298+0000","last_undegraded":"2024-09-16T15:07:22.077298+0000","last_fullsized":"2024-09-16T15:07:22.077298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:09.704280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.083140+0000","last_change":"2024-09-16T15:07:22.079328+0000","last_active":"2024-09-16T15:07:22.083140+0000","last_peered":"2024-09-16T15:07:22.083140+0000","last_clean":"2024-09-16T15:07:22.083140+0000","last_became_active":"2024-09-16T15:07:22.078344+0000","last_became_peered":"2024-09-16T15:07:22.078344+0000","last_unstale":"2024-09-16T15:07:22.083140+0000","last_undegraded":"2024-09-16T15:07:22.083140+0000","last_fullsized":"2024-09-16T15:07:22.083140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:22:23.839319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.054784+0000","last_change":"2024-09-16T15:07:21.737802+0000","last_active":"2024-09-16T15:07:22.054784+0000","last_peered":"2024-09-16T15:07:22.054784+0000","last_clean":"2024-09-16T15:07:22.054784+0000","last_became_active":"2024-09-16T15:07:21.737174+0000","last_became_peered":"2024-09-16T15:07:21.737174+0000","last_unstale":"2024-09-16T15:07:22.054784+0000","last_undegraded":"2024-09-16T15:07:22.054784+0000","last_fullsized":"2024-09-16T15:07:22.054784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082942+0000","last_change":"2024-09-16T15:07:22.079075+0000","last_active":"2024-09-16T15:07:22.082942+0000","last_peered":"2024-09-16T15:07:22.082942+0000","last_clean":"2024-09-16T15:07:22.082942+0000","last_became_active":"2024-09-16T15:07:22.077520+0000","last_became_peered":"2024-09-16T15:07:22.077520+0000","last_unstale":"2024-09-16T15:07:22.082942+0000","last_undegraded":"2024-09-16T15:07:22.082942+0000","last_fullsized":"2024-09-16T15:07:22.082942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:29.976832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.065515+0000","last_change":"2024-09-16T15:07:21.739614+0000","last_active":"2024-09-16T15:07:22.065515+0000","last_peered":"2024-09-16T15:07:22.065515+0000","last_clean":"2024-09-16T15:07:22.065515+0000","last_became_active":"2024-09-16T15:07:21.739226+0000","last_became_peered":"2024-09-16T15:07:21.739226+0000","last_unstale":"2024-09-16T15:07:22.065515+0000","last_undegraded":"2024-09-16T15:07:22.065515+0000","last_fullsized":"2024-09-16T15:07:22.065515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:13.563410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.088763+0000","last_change":"2024-09-16T15:07:22.087272+0000","last_active":"2024-09-16T15:07:22.088763+0000","last_peered":"2024-09-16T15:07:22.088763+0000","last_clean":"2024-09-16T15:07:22.088763+0000","last_became_active":"2024-09-16T15:07:22.086953+0000","last_became_peered":"2024-09-16T15:07:22.086953+0000","last_unstale":"2024-09-16T15:07:22.088763+0000","last_undegraded":"2024-09-16T15:07:22.088763+0000","last_fullsized":"2024-09-16T15:07:22.088763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:34.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.103953+0000","last_change":"2024-09-16T15:07:21.733899+0000","last_active":"2024-09-16T15:07:22.103953+0000","last_peered":"2024-09-16T15:07:22.103953+0000","last_clean":"2024-09-16T15:07:22.103953+0000","last_became_active":"2024-09-16T15:07:21.730134+0000","last_became_peered":"2024-09-16T15:07:21.730134+0000","last_unstale":"2024-09-16T15:07:22.103953+0000","last_undegraded":"2024-09-16T15:07:22.103953+0000","last_fullsized":"2024-09-16T15:07:22.103953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.089504+0000","last_change":"2024-09-16T15:07:21.720627+0000","last_active":"2024-09-16T15:07:22.089504+0000","last_peered":"2024-09-16T15:07:22.089504+0000","last_clean":"2024-09-16T15:07:22.089504+0000","last_became_active":"2024-09-16T15:07:21.719903+0000","last_became_peered":"2024-09-16T15:07:21.719903+0000","last_unstale":"2024-09-16T15:07:22.089504+0000","last_undegraded":"2024-09-16T15:07:22.089504+0000","last_fullsized":"2024-09-16T15:07:22.089504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.101782+0000","last_change":"2024-09-16T15:07:22.100423+0000","last_active":"2024-09-16T15:07:22.101782+0000","last_peered":"2024-09-16T15:07:22.101782+0000","last_clean":"2024-09-16T15:07:22.101782+0000","last_became_active":"2024-09-16T15:07:22.099384+0000","last_became_peered":"2024-09-16T15:07:22.099384+0000","last_unstale":"2024-09-16T15:07:22.101782+0000","last_undegraded":"2024-09-16T15:07:22.101782+0000","last_fullsized":"2024-09-16T15:07:22.101782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:15.595055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.089783+0000","last_change":"2024-09-16T15:07:21.720105+0000","last_active":"2024-09-16T15:07:22.089783+0000","last_peered":"2024-09-16T15:07:22.089783+0000","last_clean":"2024-09-16T15:07:22.089783+0000","last_became_active":"2024-09-16T15:07:21.719254+0000","last_became_peered":"2024-09-16T15:07:21.719254+0000","last_unstale":"2024-09-16T15:07:22.089783+0000","last_undegraded":"2024-09-16T15:07:22.089783+0000","last_fullsized":"2024-09-16T15:07:22.089783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093554+0000","last_change":"2024-09-16T15:07:22.087123+0000","last_active":"2024-09-16T15:07:22.093554+0000","last_peered":"2024-09-16T15:07:22.093554+0000","last_clean":"2024-09-16T15:07:22.093554+0000","last_became_active":"2024-09-16T15:07:22.085858+0000","last_became_peered":"2024-09-16T15:07:22.085858+0000","last_unstale":"2024-09-16T15:07:22.093554+0000","last_undegraded":"2024-09-16T15:07:22.093554+0000","last_fullsized":"2024-09-16T15:07:22.093554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:43.560388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.094789+0000","last_change":"2024-09-16T15:07:21.727975+0000","last_active":"2024-09-16T15:07:22.094789+0000","last_peered":"2024-09-16T15:07:22.094789+0000","last_clean":"2024-09-16T15:07:22.094789+0000","last_became_active":"2024-09-16T15:07:21.726999+0000","last_became_peered":"2024-09-16T15:07:21.726999+0000","last_unstale":"2024-09-16T15:07:22.094789+0000","last_undegraded":"2024-09-16T15:07:22.094789+0000","last_fullsized":"2024-09-16T15:07:22.094789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:26.627634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.070346+0000","last_change":"2024-09-16T15:07:22.064842+0000","last_active":"2024-09-16T15:07:22.070346+0000","last_peered":"2024-09-16T15:07:22.070346+0000","last_clean":"2024-09-16T15:07:22.070346+0000","last_became_active":"2024-09-16T15:07:22.064502+0000","last_became_peered":"2024-09-16T15:07:22.064502+0000","last_unstale":"2024-09-16T15:07:22.070346+0000","last_undegraded":"2024-09-16T15:07:22.070346+0000","last_fullsized":"2024-09-16T15:07:22.070346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:51.922251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067309+0000","last_change":"2024-09-16T15:07:21.725945+0000","last_active":"2024-09-16T15:07:22.067309+0000","last_peered":"2024-09-16T15:07:22.067309+0000","last_clean":"2024-09-16T15:07:22.067309+0000","last_became_active":"2024-09-16T15:07:21.724839+0000","last_became_peered":"2024-09-16T15:07:21.724839+0000","last_unstale":"2024-09-16T15:07:22.067309+0000","last_undegraded":"2024-09-16T15:07:22.067309+0000","last_fullsized":"2024-09-16T15:07:22.067309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:47.932258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067191+0000","last_change":"2024-09-16T15:07:22.058944+0000","last_active":"2024-09-16T15:07:22.067191+0000","last_peered":"2024-09-16T15:07:22.067191+0000","last_clean":"2024-09-16T15:07:22.067191+0000","last_became_active":"2024-09-16T15:07:22.058700+0000","last_became_peered":"2024-09-16T15:07:22.058700+0000","last_unstale":"2024-09-16T15:07:22.067191+0000","last_undegraded":"2024-09-16T15:07:22.067191+0000","last_fullsized":"2024-09-16T15:07:22.067191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:10.707281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.074624+0000","last_change":"2024-09-16T15:07:21.725144+0000","last_active":"2024-09-16T15:07:22.074624+0000","last_peered":"2024-09-16T15:07:22.074624+0000","last_clean":"2024-09-16T15:07:22.074624+0000","last_became_active":"2024-09-16T15:07:21.724855+0000","last_became_peered":"2024-09-16T15:07:21.724855+0000","last_unstale":"2024-09-16T15:07:22.074624+0000","last_undegraded":"2024-09-16T15:07:22.074624+0000","last_fullsized":"2024-09-16T15:07:22.074624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093365+0000","last_change":"2024-09-16T15:07:22.087348+0000","last_active":"2024-09-16T15:07:22.093365+0000","last_peered":"2024-09-16T15:07:22.093365+0000","last_clean":"2024-09-16T15:07:22.093365+0000","last_became_active":"2024-09-16T15:07:22.086906+0000","last_became_peered":"2024-09-16T15:07:22.086906+0000","last_unstale":"2024-09-16T15:07:22.093365+0000","last_undegraded":"2024-09-16T15:07:22.093365+0000","last_fullsized":"2024-09-16T15:07:22.093365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:53.121258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067004+0000","last_change":"2024-09-16T15:07:21.727272+0000","last_active":"2024-09-16T15:07:22.067004+0000","last_peered":"2024-09-16T15:07:22.067004+0000","last_clean":"2024-09-16T15:07:22.067004+0000","last_became_active":"2024-09-16T15:07:21.724280+0000","last_became_peered":"2024-09-16T15:07:21.724280+0000","last_unstale":"2024-09-16T15:07:22.067004+0000","last_undegraded":"2024-09-16T15:07:22.067004+0000","last_fullsized":"2024-09-16T15:07:22.067004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:50.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.096738+0000","last_change":"2024-09-16T15:07:22.094628+0000","last_active":"2024-09-16T15:07:22.096738+0000","last_peered":"2024-09-16T15:07:22.096738+0000","last_clean":"2024-09-16T15:07:22.096738+0000","last_became_active":"2024-09-16T15:07:22.094356+0000","last_became_peered":"2024-09-16T15:07:22.094356+0000","last_unstale":"2024-09-16T15:07:22.096738+0000","last_undegraded":"2024-09-16T15:07:22.096738+0000","last_fullsized":"2024-09-16T15:07:22.096738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:37.486620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.065647+0000","last_change":"2024-09-16T15:07:21.742014+0000","last_active":"2024-09-16T15:07:22.065647+0000","last_peered":"2024-09-16T15:07:22.065647+0000","last_clean":"2024-09-16T15:07:22.065647+0000","last_became_active":"2024-09-16T15:07:21.741361+0000","last_became_peered":"2024-09-16T15:07:21.741361+0000","last_unstale":"2024-09-16T15:07:22.065647+0000","last_undegraded":"2024-09-16T15:07:22.065647+0000","last_fullsized":"2024-09-16T15:07:22.065647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:05.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.074446+0000","last_change":"2024-09-16T15:07:22.072156+0000","last_active":"2024-09-16T15:07:22.074446+0000","last_peered":"2024-09-16T15:07:22.074446+0000","last_clean":"2024-09-16T15:07:22.074446+0000","last_became_active":"2024-09-16T15:07:22.071767+0000","last_became_peered":"2024-09-16T15:07:22.071767+0000","last_unstale":"2024-09-16T15:07:22.074446+0000","last_undegraded":"2024-09-16T15:07:22.074446+0000","last_fullsized":"2024-09-16T15:07:22.074446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:55.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.064738+0000","last_change":"2024-09-16T15:07:21.726805+0000","last_active":"2024-09-16T15:07:22.064738+0000","last_peered":"2024-09-16T15:07:22.064738+0000","last_clean":"2024-09-16T15:07:22.064738+0000","last_became_active":"2024-09-16T15:07:21.726154+0000","last_became_peered":"2024-09-16T15:07:21.726154+0000","last_unstale":"2024-09-16T15:07:22.064738+0000","last_undegraded":"2024-09-16T15:07:22.064738+0000","last_fullsized":"2024-09-16T15:07:22.064738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:21.077487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093655+0000","last_change":"2024-09-16T15:07:22.088691+0000","last_active":"2024-09-16T15:07:22.093655+0000","last_peered":"2024-09-16T15:07:22.093655+0000","last_clean":"2024-09-16T15:07:22.093655+0000","last_became_active":"2024-09-16T15:07:22.087008+0000","last_became_peered":"2024-09-16T15:07:22.087008+0000","last_unstale":"2024-09-16T15:07:22.093655+0000","last_undegraded":"2024-09-16T15:07:22.093655+0000","last_fullsized":"2024-09-16T15:07:22.093655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:07.641636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":166,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.704814+0000","last_change":"2024-09-16T15:07:11.063383+0000","last_active":"2024-09-16T15:07:21.704814+0000","last_peered":"2024-09-16T15:07:21.704814+0000","last_clean":"2024-09-16T15:07:21.704814+0000","last_became_active":"2024-09-16T15:07:11.062607+0000","last_became_peered":"2024-09-16T15:07:11.062607+0000","last_unstale":"2024-09-16T15:07:21.704814+0000","last_undegraded":"2024-09-16T15:07:21.704814+0000","last_fullsized":"2024-09-16T15:07:21.704814+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:09.570910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:09.570910+0000","last_clean_scrub_stamp":"2024-09-16T15:07:09.570910+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:23.466827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.707553+0000","last_change":"2024-09-16T15:07:17.669610+0000","last_active":"2024-09-16T15:07:21.707553+0000","last_peered":"2024-09-16T15:07:21.707553+0000","last_clean":"2024-09-16T15:07:21.707553+0000","last_became_active":"2024-09-16T15:07:15.657618+0000","last_became_peered":"2024-09-16T15:07:15.657618+0000","last_unstale":"2024-09-16T15:07:21.707553+0000","last_undegraded":"2024-09-16T15:07:21.707553+0000","last_fullsized":"2024-09-16T15:07:21.707553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:28:19.381043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085582900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067153+0000","last_change":"2024-09-16T15:07:21.741970+0000","last_active":"2024-09-16T15:07:22.067153+0000","last_peered":"2024-09-16T15:07:22.067153+0000","last_clean":"2024-09-16T15:07:22.067153+0000","last_became_active":"2024-09-16T15:07:21.741305+0000","last_became_peered":"2024-09-16T15:07:21.741305+0000","last_unstale":"2024-09-16T15:07:22.067153+0000","last_undegraded":"2024-09-16T15:07:22.067153+0000","last_fullsized":"2024-09-16T15:07:22.067153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:46.566147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.096998+0000","last_change":"2024-09-16T15:07:22.095031+0000","last_active":"2024-09-16T15:07:22.096998+0000","last_peered":"2024-09-16T15:07:22.096998+0000","last_clean":"2024-09-16T15:07:22.096998+0000","last_became_active":"2024-09-16T15:07:22.094793+0000","last_became_peered":"2024-09-16T15:07:22.094793+0000","last_unstale":"2024-09-16T15:07:22.096998+0000","last_undegraded":"2024-09-16T15:07:22.096998+0000","last_fullsized":"2024-09-16T15:07:22.096998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:38:36.377299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.703762+0000","last_change":"2024-09-16T15:07:17.669433+0000","last_active":"2024-09-16T15:07:21.703762+0000","last_peered":"2024-09-16T15:07:21.703762+0000","last_clean":"2024-09-16T15:07:21.703762+0000","last_became_active":"2024-09-16T15:07:15.656237+0000","last_became_peered":"2024-09-16T15:07:15.656237+0000","last_unstale":"2024-09-16T15:07:21.703762+0000","last_undegraded":"2024-09-16T15:07:21.703762+0000","last_fullsized":"2024-09-16T15:07:21.703762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:49.124474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068265100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.055637+0000","last_change":"2024-09-16T15:07:21.713506+0000","last_active":"2024-09-16T15:07:22.055637+0000","last_peered":"2024-09-16T15:07:22.055637+0000","last_clean":"2024-09-16T15:07:22.055637+0000","last_became_active":"2024-09-16T15:07:21.713226+0000","last_became_peered":"2024-09-16T15:07:21.713226+0000","last_unstale":"2024-09-16T15:07:22.055637+0000","last_undegraded":"2024-09-16T15:07:22.055637+0000","last_fullsized":"2024-09-16T15:07:22.055637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082923+0000","last_change":"2024-09-16T15:07:22.079061+0000","last_active":"2024-09-16T15:07:22.082923+0000","last_peered":"2024-09-16T15:07:22.082923+0000","last_clean":"2024-09-16T15:07:22.082923+0000","last_became_active":"2024-09-16T15:07:22.076697+0000","last_became_peered":"2024-09-16T15:07:22.076697+0000","last_unstale":"2024-09-16T15:07:22.082923+0000","last_undegraded":"2024-09-16T15:07:22.082923+0000","last_fullsized":"2024-09-16T15:07:22.082923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:58.996879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.718834+0000","last_change":"2024-09-16T15:07:18.173036+0000","last_active":"2024-09-16T15:07:21.718834+0000","last_peered":"2024-09-16T15:07:21.718834+0000","last_clean":"2024-09-16T15:07:21.718834+0000","last_became_active":"2024-09-16T15:07:15.656238+0000","last_became_peered":"2024-09-16T15:07:15.656238+0000","last_unstale":"2024-09-16T15:07:21.718834+0000","last_undegraded":"2024-09-16T15:07:21.718834+0000","last_fullsized":"2024-09-16T15:07:21.718834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:15:02.977857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080979399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.053789+0000","last_change":"2024-09-16T15:07:21.722652+0000","last_active":"2024-09-16T15:07:22.053789+0000","last_peered":"2024-09-16T15:07:22.053789+0000","last_clean":"2024-09-16T15:07:22.053789+0000","last_became_active":"2024-09-16T15:07:21.722308+0000","last_became_peered":"2024-09-16T15:07:21.722308+0000","last_unstale":"2024-09-16T15:07:22.053789+0000","last_undegraded":"2024-09-16T15:07:22.053789+0000","last_fullsized":"2024-09-16T15:07:22.053789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.088775+0000","last_change":"2024-09-16T15:07:22.087268+0000","last_active":"2024-09-16T15:07:22.088775+0000","last_peered":"2024-09-16T15:07:22.088775+0000","last_clean":"2024-09-16T15:07:22.088775+0000","last_became_active":"2024-09-16T15:07:22.086932+0000","last_became_peered":"2024-09-16T15:07:22.086932+0000","last_unstale":"2024-09-16T15:07:22.088775+0000","last_undegraded":"2024-09-16T15:07:22.088775+0000","last_fullsized":"2024-09-16T15:07:22.088775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:55.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.705483+0000","last_change":"2024-09-16T15:07:17.671010+0000","last_active":"2024-09-16T15:07:21.705483+0000","last_peered":"2024-09-16T15:07:21.705483+0000","last_clean":"2024-09-16T15:07:21.705483+0000","last_became_active":"2024-09-16T15:07:15.658625+0000","last_became_peered":"2024-09-16T15:07:15.658625+0000","last_unstale":"2024-09-16T15:07:21.705483+0000","last_undegraded":"2024-09-16T15:07:21.705483+0000","last_fullsized":"2024-09-16T15:07:21.705483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:18.695305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00224435,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.055332+0000","last_change":"2024-09-16T15:07:21.712013+0000","last_active":"2024-09-16T15:07:22.055332+0000","last_peered":"2024-09-16T15:07:22.055332+0000","last_clean":"2024-09-16T15:07:22.055332+0000","last_became_active":"2024-09-16T15:07:21.710551+0000","last_became_peered":"2024-09-16T15:07:21.710551+0000","last_unstale":"2024-09-16T15:07:22.055332+0000","last_undegraded":"2024-09-16T15:07:22.055332+0000","last_fullsized":"2024-09-16T15:07:22.055332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.069566+0000","last_change":"2024-09-16T15:07:22.067593+0000","last_active":"2024-09-16T15:07:22.069566+0000","last_peered":"2024-09-16T15:07:22.069566+0000","last_clean":"2024-09-16T15:07:22.069566+0000","last_became_active":"2024-09-16T15:07:22.067254+0000","last_became_peered":"2024-09-16T15:07:22.067254+0000","last_unstale":"2024-09-16T15:07:22.069566+0000","last_undegraded":"2024-09-16T15:07:22.069566+0000","last_fullsized":"2024-09-16T15:07:22.069566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:43.560388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.054647+0000","last_change":"2024-09-16T15:07:21.711346+0000","last_active":"2024-09-16T15:07:22.054647+0000","last_peered":"2024-09-16T15:07:22.054647+0000","last_clean":"2024-09-16T15:07:22.054647+0000","last_became_active":"2024-09-16T15:07:21.710693+0000","last_became_peered":"2024-09-16T15:07:21.710693+0000","last_unstale":"2024-09-16T15:07:22.054647+0000","last_undegraded":"2024-09-16T15:07:22.054647+0000","last_fullsized":"2024-09-16T15:07:22.054647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:50.814161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.685865+0000","last_change":"2024-09-16T15:07:17.664832+0000","last_active":"2024-09-16T15:07:21.685865+0000","last_peered":"2024-09-16T15:07:21.685865+0000","last_clean":"2024-09-16T15:07:21.685865+0000","last_became_active":"2024-09-16T15:07:15.654955+0000","last_became_peered":"2024-09-16T15:07:15.654955+0000","last_unstale":"2024-09-16T15:07:21.685865+0000","last_undegraded":"2024-09-16T15:07:21.685865+0000","last_fullsized":"2024-09-16T15:07:21.685865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:48:38.399733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085315100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097557+0000","last_change":"2024-09-16T15:07:22.091087+0000","last_active":"2024-09-16T15:07:22.097557+0000","last_peered":"2024-09-16T15:07:22.097557+0000","last_clean":"2024-09-16T15:07:22.097557+0000","last_became_active":"2024-09-16T15:07:22.088585+0000","last_became_peered":"2024-09-16T15:07:22.088585+0000","last_unstale":"2024-09-16T15:07:22.097557+0000","last_undegraded":"2024-09-16T15:07:22.097557+0000","last_fullsized":"2024-09-16T15:07:22.097557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:19.377720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.055132+0000","last_change":"2024-09-16T15:07:21.713440+0000","last_active":"2024-09-16T15:07:22.055132+0000","last_peered":"2024-09-16T15:07:22.055132+0000","last_clean":"2024-09-16T15:07:22.055132+0000","last_became_active":"2024-09-16T15:07:21.713127+0000","last_became_peered":"2024-09-16T15:07:21.713127+0000","last_unstale":"2024-09-16T15:07:22.055132+0000","last_undegraded":"2024-09-16T15:07:22.055132+0000","last_fullsized":"2024-09-16T15:07:22.055132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.714217+0000","last_change":"2024-09-16T15:07:17.671471+0000","last_active":"2024-09-16T15:07:21.714217+0000","last_peered":"2024-09-16T15:07:21.714217+0000","last_clean":"2024-09-16T15:07:21.714217+0000","last_became_active":"2024-09-16T15:07:15.656288+0000","last_became_peered":"2024-09-16T15:07:15.656288+0000","last_unstale":"2024-09-16T15:07:21.714217+0000","last_undegraded":"2024-09-16T15:07:21.714217+0000","last_fullsized":"2024-09-16T15:07:21.714217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:03.700310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026747759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.096370+0000","last_change":"2024-09-16T15:07:22.087230+0000","last_active":"2024-09-16T15:07:22.096370+0000","last_peered":"2024-09-16T15:07:22.096370+0000","last_clean":"2024-09-16T15:07:22.096370+0000","last_became_active":"2024-09-16T15:07:22.086917+0000","last_became_peered":"2024-09-16T15:07:22.086917+0000","last_unstale":"2024-09-16T15:07:22.096370+0000","last_undegraded":"2024-09-16T15:07:22.096370+0000","last_fullsized":"2024-09-16T15:07:22.096370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:05.650468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.077952+0000","last_change":"2024-09-16T15:07:21.727506+0000","last_active":"2024-09-16T15:07:22.077952+0000","last_peered":"2024-09-16T15:07:22.077952+0000","last_clean":"2024-09-16T15:07:22.077952+0000","last_became_active":"2024-09-16T15:07:21.724965+0000","last_became_peered":"2024-09-16T15:07:21.724965+0000","last_unstale":"2024-09-16T15:07:22.077952+0000","last_undegraded":"2024-09-16T15:07:22.077952+0000","last_fullsized":"2024-09-16T15:07:22.077952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:41.835469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.686016+0000","last_change":"2024-09-16T15:07:17.664973+0000","last_active":"2024-09-16T15:07:21.686016+0000","last_peered":"2024-09-16T15:07:21.686016+0000","last_clean":"2024-09-16T15:07:21.686016+0000","last_became_active":"2024-09-16T15:07:15.655047+0000","last_became_peered":"2024-09-16T15:07:15.655047+0000","last_unstale":"2024-09-16T15:07:21.686016+0000","last_undegraded":"2024-09-16T15:07:21.686016+0000","last_fullsized":"2024-09-16T15:07:21.686016+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:53.308490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081824399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.103422+0000","last_change":"2024-09-16T15:07:22.100811+0000","last_active":"2024-09-16T15:07:22.103422+0000","last_peered":"2024-09-16T15:07:22.103422+0000","last_clean":"2024-09-16T15:07:22.103422+0000","last_became_active":"2024-09-16T15:07:22.099732+0000","last_became_peered":"2024-09-16T15:07:22.099732+0000","last_unstale":"2024-09-16T15:07:22.103422+0000","last_undegraded":"2024-09-16T15:07:22.103422+0000","last_fullsized":"2024-09-16T15:07:22.103422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:36.011335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.096463+0000","last_change":"2024-09-16T15:07:21.733464+0000","last_active":"2024-09-16T15:07:22.096463+0000","last_peered":"2024-09-16T15:07:22.096463+0000","last_clean":"2024-09-16T15:07:22.096463+0000","last_became_active":"2024-09-16T15:07:21.732659+0000","last_became_peered":"2024-09-16T15:07:21.732659+0000","last_unstale":"2024-09-16T15:07:22.096463+0000","last_undegraded":"2024-09-16T15:07:22.096463+0000","last_fullsized":"2024-09-16T15:07:22.096463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.704487+0000","last_change":"2024-09-16T15:07:17.671161+0000","last_active":"2024-09-16T15:07:21.704487+0000","last_peered":"2024-09-16T15:07:21.704487+0000","last_clean":"2024-09-16T15:07:21.704487+0000","last_became_active":"2024-09-16T15:07:15.654237+0000","last_became_peered":"2024-09-16T15:07:15.654237+0000","last_unstale":"2024-09-16T15:07:21.704487+0000","last_undegraded":"2024-09-16T15:07:21.704487+0000","last_fullsized":"2024-09-16T15:07:21.704487+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:56.780961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356239,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.075216+0000","last_change":"2024-09-16T15:07:22.071750+0000","last_active":"2024-09-16T15:07:22.075216+0000","last_peered":"2024-09-16T15:07:22.075216+0000","last_clean":"2024-09-16T15:07:22.075216+0000","last_became_active":"2024-09-16T15:07:22.071469+0000","last_became_peered":"2024-09-16T15:07:22.071469+0000","last_unstale":"2024-09-16T15:07:22.075216+0000","last_undegraded":"2024-09-16T15:07:22.075216+0000","last_fullsized":"2024-09-16T15:07:22.075216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:45.906789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.084182+0000","last_change":"2024-09-16T15:07:22.080700+0000","last_active":"2024-09-16T15:07:22.084182+0000","last_peered":"2024-09-16T15:07:22.084182+0000","last_clean":"2024-09-16T15:07:22.084182+0000","last_became_active":"2024-09-16T15:07:22.080300+0000","last_became_peered":"2024-09-16T15:07:22.080300+0000","last_unstale":"2024-09-16T15:07:22.084182+0000","last_undegraded":"2024-09-16T15:07:22.084182+0000","last_fullsized":"2024-09-16T15:07:22.084182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:27.703305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.096828+0000","last_change":"2024-09-16T15:07:21.741959+0000","last_active":"2024-09-16T15:07:22.096828+0000","last_peered":"2024-09-16T15:07:22.096828+0000","last_clean":"2024-09-16T15:07:22.096828+0000","last_became_active":"2024-09-16T15:07:21.741329+0000","last_became_peered":"2024-09-16T15:07:21.741329+0000","last_unstale":"2024-09-16T15:07:22.096828+0000","last_undegraded":"2024-09-16T15:07:22.096828+0000","last_fullsized":"2024-09-16T15:07:22.096828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:31.395563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097512+0000","last_change":"2024-09-16T15:07:22.088939+0000","last_active":"2024-09-16T15:07:22.097512+0000","last_peered":"2024-09-16T15:07:22.097512+0000","last_clean":"2024-09-16T15:07:22.097512+0000","last_became_active":"2024-09-16T15:07:22.087096+0000","last_became_peered":"2024-09-16T15:07:22.087096+0000","last_unstale":"2024-09-16T15:07:22.097512+0000","last_undegraded":"2024-09-16T15:07:22.097512+0000","last_fullsized":"2024-09-16T15:07:22.097512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:49.184359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.080521+0000","last_change":"2024-09-16T15:07:21.737806+0000","last_active":"2024-09-16T15:07:22.080521+0000","last_peered":"2024-09-16T15:07:22.080521+0000","last_clean":"2024-09-16T15:07:22.080521+0000","last_became_active":"2024-09-16T15:07:21.737289+0000","last_became_peered":"2024-09-16T15:07:21.737289+0000","last_unstale":"2024-09-16T15:07:22.080521+0000","last_undegraded":"2024-09-16T15:07:22.080521+0000","last_fullsized":"2024-09-16T15:07:22.080521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:15.691812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.055771+0000","last_change":"2024-09-16T15:07:21.738040+0000","last_active":"2024-09-16T15:07:22.055771+0000","last_peered":"2024-09-16T15:07:22.055771+0000","last_clean":"2024-09-16T15:07:22.055771+0000","last_became_active":"2024-09-16T15:07:21.737276+0000","last_became_peered":"2024-09-16T15:07:21.737276+0000","last_unstale":"2024-09-16T15:07:22.055771+0000","last_undegraded":"2024-09-16T15:07:22.055771+0000","last_fullsized":"2024-09-16T15:07:22.055771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.074708+0000","last_change":"2024-09-16T15:07:22.073415+0000","last_active":"2024-09-16T15:07:22.074708+0000","last_peered":"2024-09-16T15:07:22.074708+0000","last_clean":"2024-09-16T15:07:22.074708+0000","last_became_active":"2024-09-16T15:07:22.073160+0000","last_became_peered":"2024-09-16T15:07:22.073160+0000","last_unstale":"2024-09-16T15:07:22.074708+0000","last_undegraded":"2024-09-16T15:07:22.074708+0000","last_fullsized":"2024-09-16T15:07:22.074708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:32.398564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.095864+0000","last_change":"2024-09-16T15:07:21.730081+0000","last_active":"2024-09-16T15:07:22.095864+0000","last_peered":"2024-09-16T15:07:22.095864+0000","last_clean":"2024-09-16T15:07:22.095864+0000","last_became_active":"2024-09-16T15:07:21.729450+0000","last_became_peered":"2024-09-16T15:07:21.729450+0000","last_unstale":"2024-09-16T15:07:22.095864+0000","last_undegraded":"2024-09-16T15:07:22.095864+0000","last_fullsized":"2024-09-16T15:07:22.095864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.101762+0000","last_change":"2024-09-16T15:07:22.100315+0000","last_active":"2024-09-16T15:07:22.101762+0000","last_peered":"2024-09-16T15:07:22.101762+0000","last_clean":"2024-09-16T15:07:22.101762+0000","last_became_active":"2024-09-16T15:07:22.099367+0000","last_became_peered":"2024-09-16T15:07:22.099367+0000","last_unstale":"2024-09-16T15:07:22.101762+0000","last_undegraded":"2024-09-16T15:07:22.101762+0000","last_fullsized":"2024-09-16T15:07:22.101762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:45.128492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082965+0000","last_change":"2024-09-16T15:07:22.079005+0000","last_active":"2024-09-16T15:07:22.082965+0000","last_peered":"2024-09-16T15:07:22.082965+0000","last_clean":"2024-09-16T15:07:22.082965+0000","last_became_active":"2024-09-16T15:07:22.077888+0000","last_became_peered":"2024-09-16T15:07:22.077888+0000","last_unstale":"2024-09-16T15:07:22.082965+0000","last_undegraded":"2024-09-16T15:07:22.082965+0000","last_fullsized":"2024-09-16T15:07:22.082965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:21.103662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.064628+0000","last_change":"2024-09-16T15:07:21.724826+0000","last_active":"2024-09-16T15:07:22.064628+0000","last_peered":"2024-09-16T15:07:22.064628+0000","last_clean":"2024-09-16T15:07:22.064628+0000","last_became_active":"2024-09-16T15:07:21.724152+0000","last_became_peered":"2024-09-16T15:07:21.724152+0000","last_unstale":"2024-09-16T15:07:22.064628+0000","last_undegraded":"2024-09-16T15:07:22.064628+0000","last_fullsized":"2024-09-16T15:07:22.064628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:26.700305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.088717+0000","last_change":"2024-09-16T15:07:22.087018+0000","last_active":"2024-09-16T15:07:22.088717+0000","last_peered":"2024-09-16T15:07:22.088717+0000","last_clean":"2024-09-16T15:07:22.088717+0000","last_became_active":"2024-09-16T15:07:22.086746+0000","last_became_peered":"2024-09-16T15:07:22.086746+0000","last_unstale":"2024-09-16T15:07:22.088717+0000","last_undegraded":"2024-09-16T15:07:22.088717+0000","last_fullsized":"2024-09-16T15:07:22.088717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:11.167685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093649+0000","last_change":"2024-09-16T15:07:21.726511+0000","last_active":"2024-09-16T15:07:22.093649+0000","last_peered":"2024-09-16T15:07:22.093649+0000","last_clean":"2024-09-16T15:07:22.093649+0000","last_became_active":"2024-09-16T15:07:21.725971+0000","last_became_peered":"2024-09-16T15:07:21.725971+0000","last_unstale":"2024-09-16T15:07:22.093649+0000","last_undegraded":"2024-09-16T15:07:22.093649+0000","last_fullsized":"2024-09-16T15:07:22.093649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:15.691812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.095299+0000","last_change":"2024-09-16T15:07:22.093131+0000","last_active":"2024-09-16T15:07:22.095299+0000","last_peered":"2024-09-16T15:07:22.095299+0000","last_clean":"2024-09-16T15:07:22.095299+0000","last_became_active":"2024-09-16T15:07:22.092810+0000","last_became_peered":"2024-09-16T15:07:22.092810+0000","last_unstale":"2024-09-16T15:07:22.095299+0000","last_undegraded":"2024-09-16T15:07:22.095299+0000","last_fullsized":"2024-09-16T15:07:22.095299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:27.872651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.100731+0000","last_change":"2024-09-16T15:07:21.709700+0000","last_active":"2024-09-16T15:07:22.100731+0000","last_peered":"2024-09-16T15:07:22.100731+0000","last_clean":"2024-09-16T15:07:22.100731+0000","last_became_active":"2024-09-16T15:07:21.708187+0000","last_became_peered":"2024-09-16T15:07:21.708187+0000","last_unstale":"2024-09-16T15:07:22.100731+0000","last_undegraded":"2024-09-16T15:07:22.100731+0000","last_fullsized":"2024-09-16T15:07:22.100731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:50.814161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.069607+0000","last_change":"2024-09-16T15:07:22.067775+0000","last_active":"2024-09-16T15:07:22.069607+0000","last_peered":"2024-09-16T15:07:22.069607+0000","last_clean":"2024-09-16T15:07:22.069607+0000","last_became_active":"2024-09-16T15:07:22.067259+0000","last_became_peered":"2024-09-16T15:07:22.067259+0000","last_unstale":"2024-09-16T15:07:22.069607+0000","last_undegraded":"2024-09-16T15:07:22.069607+0000","last_fullsized":"2024-09-16T15:07:22.069607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:04.412982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.092710+0000","last_change":"2024-09-16T15:07:21.726917+0000","last_active":"2024-09-16T15:07:22.092710+0000","last_peered":"2024-09-16T15:07:22.092710+0000","last_clean":"2024-09-16T15:07:22.092710+0000","last_became_active":"2024-09-16T15:07:21.724806+0000","last_became_peered":"2024-09-16T15:07:21.724806+0000","last_unstale":"2024-09-16T15:07:22.092710+0000","last_undegraded":"2024-09-16T15:07:22.092710+0000","last_fullsized":"2024-09-16T15:07:22.092710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:54.201635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067214+0000","last_change":"2024-09-16T15:07:22.059433+0000","last_active":"2024-09-16T15:07:22.067214+0000","last_peered":"2024-09-16T15:07:22.067214+0000","last_clean":"2024-09-16T15:07:22.067214+0000","last_became_active":"2024-09-16T15:07:22.058973+0000","last_became_peered":"2024-09-16T15:07:22.058973+0000","last_unstale":"2024-09-16T15:07:22.067214+0000","last_undegraded":"2024-09-16T15:07:22.067214+0000","last_fullsized":"2024-09-16T15:07:22.067214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:11.167685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.101717+0000","last_change":"2024-09-16T15:07:21.726761+0000","last_active":"2024-09-16T15:07:22.101717+0000","last_peered":"2024-09-16T15:07:22.101717+0000","last_clean":"2024-09-16T15:07:22.101717+0000","last_became_active":"2024-09-16T15:07:21.726186+0000","last_became_peered":"2024-09-16T15:07:21.726186+0000","last_unstale":"2024-09-16T15:07:22.101717+0000","last_undegraded":"2024-09-16T15:07:22.101717+0000","last_fullsized":"2024-09-16T15:07:22.101717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.066230+0000","last_change":"2024-09-16T15:07:22.059781+0000","last_active":"2024-09-16T15:07:22.066230+0000","last_peered":"2024-09-16T15:07:22.066230+0000","last_clean":"2024-09-16T15:07:22.066230+0000","last_became_active":"2024-09-16T15:07:22.059423+0000","last_became_peered":"2024-09-16T15:07:22.059423+0000","last_unstale":"2024-09-16T15:07:22.066230+0000","last_undegraded":"2024-09-16T15:07:22.066230+0000","last_fullsized":"2024-09-16T15:07:22.066230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:22.080488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.078150+0000","last_change":"2024-09-16T15:07:21.743147+0000","last_active":"2024-09-16T15:07:22.078150+0000","last_peered":"2024-09-16T15:07:22.078150+0000","last_clean":"2024-09-16T15:07:22.078150+0000","last_became_active":"2024-09-16T15:07:21.742365+0000","last_became_peered":"2024-09-16T15:07:21.742365+0000","last_unstale":"2024-09-16T15:07:22.078150+0000","last_undegraded":"2024-09-16T15:07:22.078150+0000","last_fullsized":"2024-09-16T15:07:22.078150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:10.164684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.083121+0000","last_change":"2024-09-16T15:07:22.076343+0000","last_active":"2024-09-16T15:07:22.083121+0000","last_peered":"2024-09-16T15:07:22.083121+0000","last_clean":"2024-09-16T15:07:22.083121+0000","last_became_active":"2024-09-16T15:07:22.075982+0000","last_became_peered":"2024-09-16T15:07:22.075982+0000","last_unstale":"2024-09-16T15:07:22.083121+0000","last_undegraded":"2024-09-16T15:07:22.083121+0000","last_fullsized":"2024-09-16T15:07:22.083121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:25:32.065204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067836+0000","last_change":"2024-09-16T15:07:21.722589+0000","last_active":"2024-09-16T15:07:22.067836+0000","last_peered":"2024-09-16T15:07:22.067836+0000","last_clean":"2024-09-16T15:07:22.067836+0000","last_became_active":"2024-09-16T15:07:21.722226+0000","last_became_peered":"2024-09-16T15:07:21.722226+0000","last_unstale":"2024-09-16T15:07:22.067836+0000","last_undegraded":"2024-09-16T15:07:22.067836+0000","last_fullsized":"2024-09-16T15:07:22.067836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.066275+0000","last_change":"2024-09-16T15:07:22.061731+0000","last_active":"2024-09-16T15:07:22.066275+0000","last_peered":"2024-09-16T15:07:22.066275+0000","last_clean":"2024-09-16T15:07:22.066275+0000","last_became_active":"2024-09-16T15:07:22.061419+0000","last_became_peered":"2024-09-16T15:07:22.061419+0000","last_unstale":"2024-09-16T15:07:22.066275+0000","last_undegraded":"2024-09-16T15:07:22.066275+0000","last_fullsized":"2024-09-16T15:07:22.066275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:47.569148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.103592+0000","last_change":"2024-09-16T15:07:21.709152+0000","last_active":"2024-09-16T15:07:22.103592+0000","last_peered":"2024-09-16T15:07:22.103592+0000","last_clean":"2024-09-16T15:07:22.103592+0000","last_became_active":"2024-09-16T15:07:21.708468+0000","last_became_peered":"2024-09-16T15:07:21.708468+0000","last_unstale":"2024-09-16T15:07:22.103592+0000","last_undegraded":"2024-09-16T15:07:22.103592+0000","last_fullsized":"2024-09-16T15:07:22.103592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":64588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":794624,"data_stored":653997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":647783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":172032,"data_stored":57434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:07:25.717 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:07:25.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.866+0000 7fe67e388640 1 Processor -- start 2024-09-16T15:07:25.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.866+0000 7fe67e388640 1 -- start start 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe67809e490 0x7fe67806afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a1010 0x7fe67814bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67806a8f0 con 0x7fe6780a2cf0 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6780a5bd0 con 0x7fe67809e490 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780a4cf0 con 0x7fe6780a1010 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe6777fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe67809e490 0x7fe67806afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe677fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe6780a2cf0 0x7fe6780a53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52556/0 (socket says 172.21.15.133:52556) 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe677fff640 1 -- 172.21.15.133:0/554659431 learned_addr learned my addr 172.21.15.133:0/554659431 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67c925640 1 --2- 172.21.15.133:0/554659431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a1010 0x7fe67814bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67c925640 1 -- 172.21.15.133:0/554659431 >> v2:172.21.15.133:3300/0 conn(0x7fe67809e490 msgr2=0x7fe67806afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67c925640 1 --2- 172.21.15.133:0/554659431 >> v2:172.21.15.133:3300/0 conn(0x7fe67809e490 0x7fe67806afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67c925640 1 -- 172.21.15.133:0/554659431 >> v1:172.21.15.133:6789/0 conn(0x7fe6780a2cf0 legacy=0x7fe6780a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67c925640 1 -- 172.21.15.133:0/554659431 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6780a3950 con 0x7fe6780a1010 2024-09-16T15:07:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe676ffd640 1 -- 172.21.15.133:0/554659431 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254001054 0 0) 0x7fe67806a8f0 con 0x7fe6780a2cf0 2024-09-16T15:07:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe6777fe640 1 --2- 172.21.15.133:0/554659431 >> v2:172.21.15.133:3300/0 conn(0x7fe67809e490 0x7fe67806afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67c925640 1 --2- 172.21.15.133:0/554659431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a1010 0x7fe67814bd90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe670009d90 tx=0x7fe67001c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d80a63f7c42c3bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:25.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe676ffd640 1 -- 172.21.15.133:0/554659431 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700a5910 con 0x7fe6780a1010 2024-09-16T15:07:25.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe676ffd640 1 -- 172.21.15.133:0/554659431 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6700a5a70 con 0x7fe6780a1010 2024-09-16T15:07:25.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe676ffd640 1 -- 172.21.15.133:0/554659431 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700a5e00 con 0x7fe6780a1010 2024-09-16T15:07:25.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 -- 172.21.15.133:0/554659431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a1010 msgr2=0x7fe67814bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 --2- 172.21.15.133:0/554659431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a1010 0x7fe67814bd90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe670009d90 tx=0x7fe67001c830 comp rx=0 tx=0).stop 2024-09-16T15:07:25.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 -- 172.21.15.133:0/554659431 shutdown_connections 2024-09-16T15:07:25.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 --2- 172.21.15.133:0/554659431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a1010 0x7fe67814bd90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 --2- 172.21.15.133:0/554659431 >> v2:172.21.15.133:3300/0 conn(0x7fe67809e490 0x7fe67806afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 -- 172.21.15.133:0/554659431 >> 172.21.15.133:0/554659431 conn(0x7fe6780a3fd0 msgr2=0x7fe6780a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:25.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 -- 172.21.15.133:0/554659431 shutdown_connections 2024-09-16T15:07:25.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.870+0000 7fe67e388640 1 -- 172.21.15.133:0/554659431 wait complete. 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 Processor -- start 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 -- start start 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe67809e490 0x7fe678098ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6780a1010 0x7fe678099540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a2cf0 0x7fe6781dd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67809a130 con 0x7fe6780a1010 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67809ad00 con 0x7fe67809e490 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67809afc0 con 0x7fe6780a2cf0 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6780a1010 0x7fe678099540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67c925640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a2cf0 0x7fe6781dd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67c925640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a2cf0 0x7fe6781dd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:34202/0 (socket says 172.21.15.133:34202) 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67c925640 1 -- 172.21.15.133:0/3726289940 learned_addr learned my addr 172.21.15.133:0/3726289940 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe677fff640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe67809e490 0x7fe678098ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67c925640 1 -- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe67809e490 msgr2=0x7fe678098ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67c925640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe67809e490 0x7fe678098ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67c925640 1 -- 172.21.15.133:0/3726289940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6780a1010 msgr2=0x7fe678099540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67c925640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6780a1010 0x7fe678099540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67c925640 1 -- 172.21.15.133:0/3726289940 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe668009420 con 0x7fe6780a2cf0 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe677fff640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe67809e490 0x7fe678098ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67c925640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a2cf0 0x7fe6781dd2c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe6700b0040 tx=0x7fe6700195d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe674ff9640 1 -- 172.21.15.133:0/3726289940 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700b9860 con 0x7fe6780a2cf0 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe674ff9640 1 -- 172.21.15.133:0/3726289940 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6700b99c0 con 0x7fe6780a2cf0 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 -- 172.21.15.133:0/3726289940 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe670009a70 con 0x7fe6780a2cf0 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe674ff9640 1 -- 172.21.15.133:0/3726289940 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700b9d70 con 0x7fe6780a2cf0 2024-09-16T15:07:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 -- 172.21.15.133:0/3726289940 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6780706a0 con 0x7fe6780a2cf0 2024-09-16T15:07:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.874+0000 7fe67e388640 1 -- 172.21.15.133:0/3726289940 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67806afe0 con 0x7fe6780a2cf0 2024-09-16T15:07:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.878+0000 7fe674ff9640 1 -- 172.21.15.133:0/3726289940 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fe6700ad440 con 0x7fe6780a2cf0 2024-09-16T15:07:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.878+0000 7fe674ff9640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.878+0000 7fe674ff9640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.878+0000 7fe674ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.878+0000 7fe674ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.878+0000 7fe674ff9640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe664097840 0x7fe664099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.878+0000 7fe674ff9640 1 -- 172.21.15.133:0/3726289940 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4362+0+0 (secure 0 0 0) 0x7fe67015b3c0 con 0x7fe6780a2cf0 2024-09-16T15:07:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.878+0000 7fe677fff640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe664097840 0x7fe664099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.878+0000 7fe677fff640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe664097840 0x7fe664099d00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe66c007fd0 tx=0x7fe66c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:25.878+0000 7fe674ff9640 1 -- 172.21.15.133:0/3726289940 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe670128ad0 con 0x7fe6780a2cf0 2024-09-16T15:07:26.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.034+0000 7fe67e388640 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2024-09-16T15:07:26.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.034+0000 7fe67e388640 1 -- 172.21.15.133:0/3726289940 --> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6781453e0 con 0x7fe664097840 2024-09-16T15:07:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.054+0000 7fe674ff9640 1 -- 172.21.15.133:0/3726289940 <== mgr.4102 v2:172.21.15.191:6824/3926506186 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271683 (secure 0 0 0) 0x7fe6781453e0 con 0x7fe664097840 2024-09-16T15:07:26.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.054+0000 7fe674ff9640 20 mgrc handle_command_reply tid 0 r 0 2024-09-16T15:07:26.060 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:26.061 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-16T15:07:26.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 10 mgrc shutdown 2024-09-16T15:07:26.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 -- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe664097840 msgr2=0x7fe664099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe664097840 0x7fe664099d00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe66c007fd0 tx=0x7fe66c009040 comp rx=0 tx=0).stop 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 -- 172.21.15.133:0/3726289940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a2cf0 msgr2=0x7fe6781dd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a2cf0 0x7fe6781dd2c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe6700b0040 tx=0x7fe6700195d0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 -- 172.21.15.133:0/3726289940 shutdown_connections 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6780a2cf0 0x7fe6781dd2c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe664097840 0x7fe664099d00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6780a1010 0x7fe678099540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 --2- 172.21.15.133:0/3726289940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe67809e490 0x7fe678098ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 -- 172.21.15.133:0/3726289940 >> 172.21.15.133:0/3726289940 conn(0x7fe6780a3fd0 msgr2=0x7fe6780b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 -- 172.21.15.133:0/3726289940 shutdown_connections 2024-09-16T15:07:26.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.058+0000 7fe67e388640 1 -- 172.21.15.133:0/3726289940 wait complete. 2024-09-16T15:07:26.079 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-16T15:07:24.690268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592749,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":205,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":109988,"kb_used_data":1892,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377377372,"statfs":{"total":386547056640,"available":386434428928,"internally_reserved":0,"allocated":1937408,"data_stored":1423802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6374,"internal_metadata":110618394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020614"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.078209+0000","last_change":"2024-09-16T15:07:22.073492+0000","last_active":"2024-09-16T15:07:22.078209+0000","last_peered":"2024-09-16T15:07:22.078209+0000","last_clean":"2024-09-16T15:07:22.078209+0000","last_became_active":"2024-09-16T15:07:22.073186+0000","last_became_peered":"2024-09-16T15:07:22.073186+0000","last_unstale":"2024-09-16T15:07:22.078209+0000","last_undegraded":"2024-09-16T15:07:22.078209+0000","last_fullsized":"2024-09-16T15:07:22.078209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.091001+0000","last_change":"2024-09-16T15:07:22.089956+0000","last_active":"2024-09-16T15:07:22.091001+0000","last_peered":"2024-09-16T15:07:22.091001+0000","last_clean":"2024-09-16T15:07:22.091001+0000","last_became_active":"2024-09-16T15:07:22.089659+0000","last_became_peered":"2024-09-16T15:07:22.089659+0000","last_unstale":"2024-09-16T15:07:22.091001+0000","last_undegraded":"2024-09-16T15:07:22.091001+0000","last_fullsized":"2024-09-16T15:07:22.091001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:14.566411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.095057+0000","last_change":"2024-09-16T15:07:22.092432+0000","last_active":"2024-09-16T15:07:22.095057+0000","last_peered":"2024-09-16T15:07:22.095057+0000","last_clean":"2024-09-16T15:07:22.095057+0000","last_became_active":"2024-09-16T15:07:22.092188+0000","last_became_peered":"2024-09-16T15:07:22.092188+0000","last_unstale":"2024-09-16T15:07:22.095057+0000","last_undegraded":"2024-09-16T15:07:22.095057+0000","last_fullsized":"2024-09-16T15:07:22.095057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:25.997418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.098144+0000","last_change":"2024-09-16T15:07:22.095720+0000","last_active":"2024-09-16T15:07:22.098144+0000","last_peered":"2024-09-16T15:07:22.098144+0000","last_clean":"2024-09-16T15:07:22.098144+0000","last_became_active":"2024-09-16T15:07:22.093626+0000","last_became_peered":"2024-09-16T15:07:22.093626+0000","last_unstale":"2024-09-16T15:07:22.098144+0000","last_undegraded":"2024-09-16T15:07:22.098144+0000","last_fullsized":"2024-09-16T15:07:22.098144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:24.351732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082976+0000","last_change":"2024-09-16T15:07:22.077679+0000","last_active":"2024-09-16T15:07:22.082976+0000","last_peered":"2024-09-16T15:07:22.082976+0000","last_clean":"2024-09-16T15:07:22.082976+0000","last_became_active":"2024-09-16T15:07:22.077387+0000","last_became_peered":"2024-09-16T15:07:22.077387+0000","last_unstale":"2024-09-16T15:07:22.082976+0000","last_undegraded":"2024-09-16T15:07:22.082976+0000","last_fullsized":"2024-09-16T15:07:22.082976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:09.497997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082851+0000","last_change":"2024-09-16T15:07:22.077568+0000","last_active":"2024-09-16T15:07:22.082851+0000","last_peered":"2024-09-16T15:07:22.082851+0000","last_clean":"2024-09-16T15:07:22.082851+0000","last_became_active":"2024-09-16T15:07:22.076638+0000","last_became_peered":"2024-09-16T15:07:22.076638+0000","last_unstale":"2024-09-16T15:07:22.082851+0000","last_undegraded":"2024-09-16T15:07:22.082851+0000","last_fullsized":"2024-09-16T15:07:22.082851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:28.264456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.081204+0000","last_change":"2024-09-16T15:07:22.075898+0000","last_active":"2024-09-16T15:07:22.081204+0000","last_peered":"2024-09-16T15:07:22.081204+0000","last_clean":"2024-09-16T15:07:22.081204+0000","last_became_active":"2024-09-16T15:07:22.075448+0000","last_became_peered":"2024-09-16T15:07:22.075448+0000","last_unstale":"2024-09-16T15:07:22.081204+0000","last_undegraded":"2024-09-16T15:07:22.081204+0000","last_fullsized":"2024-09-16T15:07:22.081204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:03.093921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.101627+0000","last_change":"2024-09-16T15:07:22.096662+0000","last_active":"2024-09-16T15:07:22.101627+0000","last_peered":"2024-09-16T15:07:22.101627+0000","last_clean":"2024-09-16T15:07:22.101627+0000","last_became_active":"2024-09-16T15:07:22.095671+0000","last_became_peered":"2024-09-16T15:07:22.095671+0000","last_unstale":"2024-09-16T15:07:22.101627+0000","last_undegraded":"2024-09-16T15:07:22.101627+0000","last_fullsized":"2024-09-16T15:07:22.101627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:47.108958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093209+0000","last_change":"2024-09-16T15:07:22.092184+0000","last_active":"2024-09-16T15:07:22.093209+0000","last_peered":"2024-09-16T15:07:22.093209+0000","last_clean":"2024-09-16T15:07:22.093209+0000","last_became_active":"2024-09-16T15:07:22.091896+0000","last_became_peered":"2024-09-16T15:07:22.091896+0000","last_unstale":"2024-09-16T15:07:22.093209+0000","last_undegraded":"2024-09-16T15:07:22.093209+0000","last_fullsized":"2024-09-16T15:07:22.093209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.098099+0000","last_change":"2024-09-16T15:07:22.093487+0000","last_active":"2024-09-16T15:07:22.098099+0000","last_peered":"2024-09-16T15:07:22.098099+0000","last_clean":"2024-09-16T15:07:22.098099+0000","last_became_active":"2024-09-16T15:07:22.091134+0000","last_became_peered":"2024-09-16T15:07:22.091134+0000","last_unstale":"2024-09-16T15:07:22.098099+0000","last_undegraded":"2024-09-16T15:07:22.098099+0000","last_fullsized":"2024-09-16T15:07:22.098099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:55.259949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097499+0000","last_change":"2024-09-16T15:07:22.091123+0000","last_active":"2024-09-16T15:07:22.097499+0000","last_peered":"2024-09-16T15:07:22.097499+0000","last_clean":"2024-09-16T15:07:22.097499+0000","last_became_active":"2024-09-16T15:07:22.090822+0000","last_became_peered":"2024-09-16T15:07:22.090822+0000","last_unstale":"2024-09-16T15:07:22.097499+0000","last_undegraded":"2024-09-16T15:07:22.097499+0000","last_fullsized":"2024-09-16T15:07:22.097499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.081454+0000","last_change":"2024-09-16T15:07:22.075137+0000","last_active":"2024-09-16T15:07:22.081454+0000","last_peered":"2024-09-16T15:07:22.081454+0000","last_clean":"2024-09-16T15:07:22.081454+0000","last_became_active":"2024-09-16T15:07:22.074779+0000","last_became_peered":"2024-09-16T15:07:22.074779+0000","last_unstale":"2024-09-16T15:07:22.081454+0000","last_undegraded":"2024-09-16T15:07:22.081454+0000","last_fullsized":"2024-09-16T15:07:22.081454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:34.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.070550+0000","last_change":"2024-09-16T15:07:22.069681+0000","last_active":"2024-09-16T15:07:22.070550+0000","last_peered":"2024-09-16T15:07:22.070550+0000","last_clean":"2024-09-16T15:07:22.070550+0000","last_became_active":"2024-09-16T15:07:22.069331+0000","last_became_peered":"2024-09-16T15:07:22.069331+0000","last_unstale":"2024-09-16T15:07:22.070550+0000","last_undegraded":"2024-09-16T15:07:22.070550+0000","last_fullsized":"2024-09-16T15:07:22.070550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:27.703305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097717+0000","last_change":"2024-09-16T15:07:22.093463+0000","last_active":"2024-09-16T15:07:22.097717+0000","last_peered":"2024-09-16T15:07:22.097717+0000","last_clean":"2024-09-16T15:07:22.097717+0000","last_became_active":"2024-09-16T15:07:22.091154+0000","last_became_peered":"2024-09-16T15:07:22.091154+0000","last_unstale":"2024-09-16T15:07:22.097717+0000","last_undegraded":"2024-09-16T15:07:22.097717+0000","last_fullsized":"2024-09-16T15:07:22.097717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:24:09.086333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093577+0000","last_change":"2024-09-16T15:07:22.087589+0000","last_active":"2024-09-16T15:07:22.093577+0000","last_peered":"2024-09-16T15:07:22.093577+0000","last_clean":"2024-09-16T15:07:22.093577+0000","last_became_active":"2024-09-16T15:07:22.087167+0000","last_became_peered":"2024-09-16T15:07:22.087167+0000","last_unstale":"2024-09-16T15:07:22.093577+0000","last_undegraded":"2024-09-16T15:07:22.093577+0000","last_fullsized":"2024-09-16T15:07:22.093577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:26.349524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.081253+0000","last_change":"2024-09-16T15:07:22.076065+0000","last_active":"2024-09-16T15:07:22.081253+0000","last_peered":"2024-09-16T15:07:22.081253+0000","last_clean":"2024-09-16T15:07:22.081253+0000","last_became_active":"2024-09-16T15:07:22.075724+0000","last_became_peered":"2024-09-16T15:07:22.075724+0000","last_unstale":"2024-09-16T15:07:22.081253+0000","last_undegraded":"2024-09-16T15:07:22.081253+0000","last_fullsized":"2024-09-16T15:07:22.081253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:20:30.555559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.099304+0000","last_change":"2024-09-16T15:07:22.091007+0000","last_active":"2024-09-16T15:07:22.099304+0000","last_peered":"2024-09-16T15:07:22.099304+0000","last_clean":"2024-09-16T15:07:22.099304+0000","last_became_active":"2024-09-16T15:07:22.089216+0000","last_became_peered":"2024-09-16T15:07:22.089216+0000","last_unstale":"2024-09-16T15:07:22.099304+0000","last_undegraded":"2024-09-16T15:07:22.099304+0000","last_fullsized":"2024-09-16T15:07:22.099304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:43.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082877+0000","last_change":"2024-09-16T15:07:22.077822+0000","last_active":"2024-09-16T15:07:22.082877+0000","last_peered":"2024-09-16T15:07:22.082877+0000","last_clean":"2024-09-16T15:07:22.082877+0000","last_became_active":"2024-09-16T15:07:22.076545+0000","last_became_peered":"2024-09-16T15:07:22.076545+0000","last_unstale":"2024-09-16T15:07:22.082877+0000","last_undegraded":"2024-09-16T15:07:22.082877+0000","last_fullsized":"2024-09-16T15:07:22.082877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:03.892835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.075184+0000","last_change":"2024-09-16T15:07:22.074503+0000","last_active":"2024-09-16T15:07:22.075184+0000","last_peered":"2024-09-16T15:07:22.075184+0000","last_clean":"2024-09-16T15:07:22.075184+0000","last_became_active":"2024-09-16T15:07:22.074240+0000","last_became_peered":"2024-09-16T15:07:22.074240+0000","last_unstale":"2024-09-16T15:07:22.075184+0000","last_undegraded":"2024-09-16T15:07:22.075184+0000","last_fullsized":"2024-09-16T15:07:22.075184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:45.906789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097702+0000","last_change":"2024-09-16T15:07:22.090412+0000","last_active":"2024-09-16T15:07:22.097702+0000","last_peered":"2024-09-16T15:07:22.097702+0000","last_clean":"2024-09-16T15:07:22.097702+0000","last_became_active":"2024-09-16T15:07:22.089178+0000","last_became_peered":"2024-09-16T15:07:22.089178+0000","last_unstale":"2024-09-16T15:07:22.097702+0000","last_undegraded":"2024-09-16T15:07:22.097702+0000","last_fullsized":"2024-09-16T15:07:22.097702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:20.559503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.088950+0000","last_change":"2024-09-16T15:07:22.085622+0000","last_active":"2024-09-16T15:07:22.088950+0000","last_peered":"2024-09-16T15:07:22.088950+0000","last_clean":"2024-09-16T15:07:22.088950+0000","last_became_active":"2024-09-16T15:07:22.085321+0000","last_became_peered":"2024-09-16T15:07:22.085321+0000","last_unstale":"2024-09-16T15:07:22.088950+0000","last_undegraded":"2024-09-16T15:07:22.088950+0000","last_fullsized":"2024-09-16T15:07:22.088950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:25.939312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093696+0000","last_change":"2024-09-16T15:07:22.088691+0000","last_active":"2024-09-16T15:07:22.093696+0000","last_peered":"2024-09-16T15:07:22.093696+0000","last_clean":"2024-09-16T15:07:22.093696+0000","last_became_active":"2024-09-16T15:07:22.085505+0000","last_became_peered":"2024-09-16T15:07:22.085505+0000","last_unstale":"2024-09-16T15:07:22.093696+0000","last_undegraded":"2024-09-16T15:07:22.093696+0000","last_fullsized":"2024-09-16T15:07:22.093696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:20.387713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.095357+0000","last_change":"2024-09-16T15:07:22.093287+0000","last_active":"2024-09-16T15:07:22.095357+0000","last_peered":"2024-09-16T15:07:22.095357+0000","last_clean":"2024-09-16T15:07:22.095357+0000","last_became_active":"2024-09-16T15:07:22.092983+0000","last_became_peered":"2024-09-16T15:07:22.092983+0000","last_unstale":"2024-09-16T15:07:22.095357+0000","last_undegraded":"2024-09-16T15:07:22.095357+0000","last_fullsized":"2024-09-16T15:07:22.095357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:32.532671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.099462+0000","last_change":"2024-09-16T15:07:22.096232+0000","last_active":"2024-09-16T15:07:22.099462+0000","last_peered":"2024-09-16T15:07:22.099462+0000","last_clean":"2024-09-16T15:07:22.099462+0000","last_became_active":"2024-09-16T15:07:22.091234+0000","last_became_peered":"2024-09-16T15:07:22.091234+0000","last_unstale":"2024-09-16T15:07:22.099462+0000","last_undegraded":"2024-09-16T15:07:22.099462+0000","last_fullsized":"2024-09-16T15:07:22.099462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:23.826942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093633+0000","last_change":"2024-09-16T15:07:22.088597+0000","last_active":"2024-09-16T15:07:22.093633+0000","last_peered":"2024-09-16T15:07:22.093633+0000","last_clean":"2024-09-16T15:07:22.093633+0000","last_became_active":"2024-09-16T15:07:22.086951+0000","last_became_peered":"2024-09-16T15:07:22.086951+0000","last_unstale":"2024-09-16T15:07:22.093633+0000","last_undegraded":"2024-09-16T15:07:22.093633+0000","last_fullsized":"2024-09-16T15:07:22.093633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:57.389973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.070114+0000","last_change":"2024-09-16T15:07:22.067847+0000","last_active":"2024-09-16T15:07:22.070114+0000","last_peered":"2024-09-16T15:07:22.070114+0000","last_clean":"2024-09-16T15:07:22.070114+0000","last_became_active":"2024-09-16T15:07:22.067337+0000","last_became_peered":"2024-09-16T15:07:22.067337+0000","last_unstale":"2024-09-16T15:07:22.070114+0000","last_undegraded":"2024-09-16T15:07:22.070114+0000","last_fullsized":"2024-09-16T15:07:22.070114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:23.826942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.083117+0000","last_change":"2024-09-16T15:07:22.079269+0000","last_active":"2024-09-16T15:07:22.083117+0000","last_peered":"2024-09-16T15:07:22.083117+0000","last_clean":"2024-09-16T15:07:22.083117+0000","last_became_active":"2024-09-16T15:07:22.078377+0000","last_became_peered":"2024-09-16T15:07:22.078377+0000","last_unstale":"2024-09-16T15:07:22.083117+0000","last_undegraded":"2024-09-16T15:07:22.083117+0000","last_fullsized":"2024-09-16T15:07:22.083117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:51.357090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.069644+0000","last_change":"2024-09-16T15:07:22.067944+0000","last_active":"2024-09-16T15:07:22.069644+0000","last_peered":"2024-09-16T15:07:22.069644+0000","last_clean":"2024-09-16T15:07:22.069644+0000","last_became_active":"2024-09-16T15:07:22.067670+0000","last_became_peered":"2024-09-16T15:07:22.067670+0000","last_unstale":"2024-09-16T15:07:22.069644+0000","last_undegraded":"2024-09-16T15:07:22.069644+0000","last_fullsized":"2024-09-16T15:07:22.069644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:10.707281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.075117+0000","last_change":"2024-09-16T15:07:22.074211+0000","last_active":"2024-09-16T15:07:22.075117+0000","last_peered":"2024-09-16T15:07:22.075117+0000","last_clean":"2024-09-16T15:07:22.075117+0000","last_became_active":"2024-09-16T15:07:22.073954+0000","last_became_peered":"2024-09-16T15:07:22.073954+0000","last_unstale":"2024-09-16T15:07:22.075117+0000","last_undegraded":"2024-09-16T15:07:22.075117+0000","last_fullsized":"2024-09-16T15:07:22.075117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:51.922251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097535+0000","last_change":"2024-09-16T15:07:22.088989+0000","last_active":"2024-09-16T15:07:22.097535+0000","last_peered":"2024-09-16T15:07:22.097535+0000","last_clean":"2024-09-16T15:07:22.097535+0000","last_became_active":"2024-09-16T15:07:22.085895+0000","last_became_peered":"2024-09-16T15:07:22.085895+0000","last_unstale":"2024-09-16T15:07:22.097535+0000","last_undegraded":"2024-09-16T15:07:22.097535+0000","last_fullsized":"2024-09-16T15:07:22.097535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:58.765327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093222+0000","last_change":"2024-09-16T15:07:22.092307+0000","last_active":"2024-09-16T15:07:22.093222+0000","last_peered":"2024-09-16T15:07:22.093222+0000","last_clean":"2024-09-16T15:07:22.093222+0000","last_became_active":"2024-09-16T15:07:22.091941+0000","last_became_peered":"2024-09-16T15:07:22.091941+0000","last_unstale":"2024-09-16T15:07:22.093222+0000","last_undegraded":"2024-09-16T15:07:22.093222+0000","last_fullsized":"2024-09-16T15:07:22.093222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:14.566411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.099446+0000","last_change":"2024-09-16T15:07:22.096217+0000","last_active":"2024-09-16T15:07:22.099446+0000","last_peered":"2024-09-16T15:07:22.099446+0000","last_clean":"2024-09-16T15:07:22.099446+0000","last_became_active":"2024-09-16T15:07:22.092983+0000","last_became_peered":"2024-09-16T15:07:22.092983+0000","last_unstale":"2024-09-16T15:07:22.099446+0000","last_undegraded":"2024-09-16T15:07:22.099446+0000","last_fullsized":"2024-09-16T15:07:22.099446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:04.412982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.083172+0000","last_change":"2024-09-16T15:07:22.079554+0000","last_active":"2024-09-16T15:07:22.083172+0000","last_peered":"2024-09-16T15:07:22.083172+0000","last_clean":"2024-09-16T15:07:22.083172+0000","last_became_active":"2024-09-16T15:07:22.078495+0000","last_became_peered":"2024-09-16T15:07:22.078495+0000","last_unstale":"2024-09-16T15:07:22.083172+0000","last_undegraded":"2024-09-16T15:07:22.083172+0000","last_fullsized":"2024-09-16T15:07:22.083172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:25.939312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.077298+0000","last_change":"2024-09-16T15:07:21.742735+0000","last_active":"2024-09-16T15:07:22.077298+0000","last_peered":"2024-09-16T15:07:22.077298+0000","last_clean":"2024-09-16T15:07:22.077298+0000","last_became_active":"2024-09-16T15:07:21.742323+0000","last_became_peered":"2024-09-16T15:07:21.742323+0000","last_unstale":"2024-09-16T15:07:22.077298+0000","last_undegraded":"2024-09-16T15:07:22.077298+0000","last_fullsized":"2024-09-16T15:07:22.077298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:09.704280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.083140+0000","last_change":"2024-09-16T15:07:22.079328+0000","last_active":"2024-09-16T15:07:22.083140+0000","last_peered":"2024-09-16T15:07:22.083140+0000","last_clean":"2024-09-16T15:07:22.083140+0000","last_became_active":"2024-09-16T15:07:22.078344+0000","last_became_peered":"2024-09-16T15:07:22.078344+0000","last_unstale":"2024-09-16T15:07:22.083140+0000","last_undegraded":"2024-09-16T15:07:22.083140+0000","last_fullsized":"2024-09-16T15:07:22.083140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:22:23.839319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.054784+0000","last_change":"2024-09-16T15:07:21.737802+0000","last_active":"2024-09-16T15:07:22.054784+0000","last_peered":"2024-09-16T15:07:22.054784+0000","last_clean":"2024-09-16T15:07:22.054784+0000","last_became_active":"2024-09-16T15:07:21.737174+0000","last_became_peered":"2024-09-16T15:07:21.737174+0000","last_unstale":"2024-09-16T15:07:22.054784+0000","last_undegraded":"2024-09-16T15:07:22.054784+0000","last_fullsized":"2024-09-16T15:07:22.054784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082942+0000","last_change":"2024-09-16T15:07:22.079075+0000","last_active":"2024-09-16T15:07:22.082942+0000","last_peered":"2024-09-16T15:07:22.082942+0000","last_clean":"2024-09-16T15:07:22.082942+0000","last_became_active":"2024-09-16T15:07:22.077520+0000","last_became_peered":"2024-09-16T15:07:22.077520+0000","last_unstale":"2024-09-16T15:07:22.082942+0000","last_undegraded":"2024-09-16T15:07:22.082942+0000","last_fullsized":"2024-09-16T15:07:22.082942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:29.976832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.065515+0000","last_change":"2024-09-16T15:07:21.739614+0000","last_active":"2024-09-16T15:07:22.065515+0000","last_peered":"2024-09-16T15:07:22.065515+0000","last_clean":"2024-09-16T15:07:22.065515+0000","last_became_active":"2024-09-16T15:07:21.739226+0000","last_became_peered":"2024-09-16T15:07:21.739226+0000","last_unstale":"2024-09-16T15:07:22.065515+0000","last_undegraded":"2024-09-16T15:07:22.065515+0000","last_fullsized":"2024-09-16T15:07:22.065515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:13.563410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.088763+0000","last_change":"2024-09-16T15:07:22.087272+0000","last_active":"2024-09-16T15:07:22.088763+0000","last_peered":"2024-09-16T15:07:22.088763+0000","last_clean":"2024-09-16T15:07:22.088763+0000","last_became_active":"2024-09-16T15:07:22.086953+0000","last_became_peered":"2024-09-16T15:07:22.086953+0000","last_unstale":"2024-09-16T15:07:22.088763+0000","last_undegraded":"2024-09-16T15:07:22.088763+0000","last_fullsized":"2024-09-16T15:07:22.088763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:34.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.103953+0000","last_change":"2024-09-16T15:07:21.733899+0000","last_active":"2024-09-16T15:07:22.103953+0000","last_peered":"2024-09-16T15:07:22.103953+0000","last_clean":"2024-09-16T15:07:22.103953+0000","last_became_active":"2024-09-16T15:07:21.730134+0000","last_became_peered":"2024-09-16T15:07:21.730134+0000","last_unstale":"2024-09-16T15:07:22.103953+0000","last_undegraded":"2024-09-16T15:07:22.103953+0000","last_fullsized":"2024-09-16T15:07:22.103953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.089504+0000","last_change":"2024-09-16T15:07:21.720627+0000","last_active":"2024-09-16T15:07:22.089504+0000","last_peered":"2024-09-16T15:07:22.089504+0000","last_clean":"2024-09-16T15:07:22.089504+0000","last_became_active":"2024-09-16T15:07:21.719903+0000","last_became_peered":"2024-09-16T15:07:21.719903+0000","last_unstale":"2024-09-16T15:07:22.089504+0000","last_undegraded":"2024-09-16T15:07:22.089504+0000","last_fullsized":"2024-09-16T15:07:22.089504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.101782+0000","last_change":"2024-09-16T15:07:22.100423+0000","last_active":"2024-09-16T15:07:22.101782+0000","last_peered":"2024-09-16T15:07:22.101782+0000","last_clean":"2024-09-16T15:07:22.101782+0000","last_became_active":"2024-09-16T15:07:22.099384+0000","last_became_peered":"2024-09-16T15:07:22.099384+0000","last_unstale":"2024-09-16T15:07:22.101782+0000","last_undegraded":"2024-09-16T15:07:22.101782+0000","last_fullsized":"2024-09-16T15:07:22.101782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:15.595055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.089783+0000","last_change":"2024-09-16T15:07:21.720105+0000","last_active":"2024-09-16T15:07:22.089783+0000","last_peered":"2024-09-16T15:07:22.089783+0000","last_clean":"2024-09-16T15:07:22.089783+0000","last_became_active":"2024-09-16T15:07:21.719254+0000","last_became_peered":"2024-09-16T15:07:21.719254+0000","last_unstale":"2024-09-16T15:07:22.089783+0000","last_undegraded":"2024-09-16T15:07:22.089783+0000","last_fullsized":"2024-09-16T15:07:22.089783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093554+0000","last_change":"2024-09-16T15:07:22.087123+0000","last_active":"2024-09-16T15:07:22.093554+0000","last_peered":"2024-09-16T15:07:22.093554+0000","last_clean":"2024-09-16T15:07:22.093554+0000","last_became_active":"2024-09-16T15:07:22.085858+0000","last_became_peered":"2024-09-16T15:07:22.085858+0000","last_unstale":"2024-09-16T15:07:22.093554+0000","last_undegraded":"2024-09-16T15:07:22.093554+0000","last_fullsized":"2024-09-16T15:07:22.093554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:43.560388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.094789+0000","last_change":"2024-09-16T15:07:21.727975+0000","last_active":"2024-09-16T15:07:22.094789+0000","last_peered":"2024-09-16T15:07:22.094789+0000","last_clean":"2024-09-16T15:07:22.094789+0000","last_became_active":"2024-09-16T15:07:21.726999+0000","last_became_peered":"2024-09-16T15:07:21.726999+0000","last_unstale":"2024-09-16T15:07:22.094789+0000","last_undegraded":"2024-09-16T15:07:22.094789+0000","last_fullsized":"2024-09-16T15:07:22.094789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:26.627634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.070346+0000","last_change":"2024-09-16T15:07:22.064842+0000","last_active":"2024-09-16T15:07:22.070346+0000","last_peered":"2024-09-16T15:07:22.070346+0000","last_clean":"2024-09-16T15:07:22.070346+0000","last_became_active":"2024-09-16T15:07:22.064502+0000","last_became_peered":"2024-09-16T15:07:22.064502+0000","last_unstale":"2024-09-16T15:07:22.070346+0000","last_undegraded":"2024-09-16T15:07:22.070346+0000","last_fullsized":"2024-09-16T15:07:22.070346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:51.922251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067309+0000","last_change":"2024-09-16T15:07:21.725945+0000","last_active":"2024-09-16T15:07:22.067309+0000","last_peered":"2024-09-16T15:07:22.067309+0000","last_clean":"2024-09-16T15:07:22.067309+0000","last_became_active":"2024-09-16T15:07:21.724839+0000","last_became_peered":"2024-09-16T15:07:21.724839+0000","last_unstale":"2024-09-16T15:07:22.067309+0000","last_undegraded":"2024-09-16T15:07:22.067309+0000","last_fullsized":"2024-09-16T15:07:22.067309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:47.932258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067191+0000","last_change":"2024-09-16T15:07:22.058944+0000","last_active":"2024-09-16T15:07:22.067191+0000","last_peered":"2024-09-16T15:07:22.067191+0000","last_clean":"2024-09-16T15:07:22.067191+0000","last_became_active":"2024-09-16T15:07:22.058700+0000","last_became_peered":"2024-09-16T15:07:22.058700+0000","last_unstale":"2024-09-16T15:07:22.067191+0000","last_undegraded":"2024-09-16T15:07:22.067191+0000","last_fullsized":"2024-09-16T15:07:22.067191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:10.707281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.074624+0000","last_change":"2024-09-16T15:07:21.725144+0000","last_active":"2024-09-16T15:07:22.074624+0000","last_peered":"2024-09-16T15:07:22.074624+0000","last_clean":"2024-09-16T15:07:22.074624+0000","last_became_active":"2024-09-16T15:07:21.724855+0000","last_became_peered":"2024-09-16T15:07:21.724855+0000","last_unstale":"2024-09-16T15:07:22.074624+0000","last_undegraded":"2024-09-16T15:07:22.074624+0000","last_fullsized":"2024-09-16T15:07:22.074624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093365+0000","last_change":"2024-09-16T15:07:22.087348+0000","last_active":"2024-09-16T15:07:22.093365+0000","last_peered":"2024-09-16T15:07:22.093365+0000","last_clean":"2024-09-16T15:07:22.093365+0000","last_became_active":"2024-09-16T15:07:22.086906+0000","last_became_peered":"2024-09-16T15:07:22.086906+0000","last_unstale":"2024-09-16T15:07:22.093365+0000","last_undegraded":"2024-09-16T15:07:22.093365+0000","last_fullsized":"2024-09-16T15:07:22.093365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:53.121258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067004+0000","last_change":"2024-09-16T15:07:21.727272+0000","last_active":"2024-09-16T15:07:22.067004+0000","last_peered":"2024-09-16T15:07:22.067004+0000","last_clean":"2024-09-16T15:07:22.067004+0000","last_became_active":"2024-09-16T15:07:21.724280+0000","last_became_peered":"2024-09-16T15:07:21.724280+0000","last_unstale":"2024-09-16T15:07:22.067004+0000","last_undegraded":"2024-09-16T15:07:22.067004+0000","last_fullsized":"2024-09-16T15:07:22.067004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:50.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.096738+0000","last_change":"2024-09-16T15:07:22.094628+0000","last_active":"2024-09-16T15:07:22.096738+0000","last_peered":"2024-09-16T15:07:22.096738+0000","last_clean":"2024-09-16T15:07:22.096738+0000","last_became_active":"2024-09-16T15:07:22.094356+0000","last_became_peered":"2024-09-16T15:07:22.094356+0000","last_unstale":"2024-09-16T15:07:22.096738+0000","last_undegraded":"2024-09-16T15:07:22.096738+0000","last_fullsized":"2024-09-16T15:07:22.096738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:37.486620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.065647+0000","last_change":"2024-09-16T15:07:21.742014+0000","last_active":"2024-09-16T15:07:22.065647+0000","last_peered":"2024-09-16T15:07:22.065647+0000","last_clean":"2024-09-16T15:07:22.065647+0000","last_became_active":"2024-09-16T15:07:21.741361+0000","last_became_peered":"2024-09-16T15:07:21.741361+0000","last_unstale":"2024-09-16T15:07:22.065647+0000","last_undegraded":"2024-09-16T15:07:22.065647+0000","last_fullsized":"2024-09-16T15:07:22.065647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:05.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.074446+0000","last_change":"2024-09-16T15:07:22.072156+0000","last_active":"2024-09-16T15:07:22.074446+0000","last_peered":"2024-09-16T15:07:22.074446+0000","last_clean":"2024-09-16T15:07:22.074446+0000","last_became_active":"2024-09-16T15:07:22.071767+0000","last_became_peered":"2024-09-16T15:07:22.071767+0000","last_unstale":"2024-09-16T15:07:22.074446+0000","last_undegraded":"2024-09-16T15:07:22.074446+0000","last_fullsized":"2024-09-16T15:07:22.074446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:55.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.064738+0000","last_change":"2024-09-16T15:07:21.726805+0000","last_active":"2024-09-16T15:07:22.064738+0000","last_peered":"2024-09-16T15:07:22.064738+0000","last_clean":"2024-09-16T15:07:22.064738+0000","last_became_active":"2024-09-16T15:07:21.726154+0000","last_became_peered":"2024-09-16T15:07:21.726154+0000","last_unstale":"2024-09-16T15:07:22.064738+0000","last_undegraded":"2024-09-16T15:07:22.064738+0000","last_fullsized":"2024-09-16T15:07:22.064738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:21.077487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093655+0000","last_change":"2024-09-16T15:07:22.088691+0000","last_active":"2024-09-16T15:07:22.093655+0000","last_peered":"2024-09-16T15:07:22.093655+0000","last_clean":"2024-09-16T15:07:22.093655+0000","last_became_active":"2024-09-16T15:07:22.087008+0000","last_became_peered":"2024-09-16T15:07:22.087008+0000","last_unstale":"2024-09-16T15:07:22.093655+0000","last_undegraded":"2024-09-16T15:07:22.093655+0000","last_fullsized":"2024-09-16T15:07:22.093655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:07.641636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":166,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.704814+0000","last_change":"2024-09-16T15:07:11.063383+0000","last_active":"2024-09-16T15:07:21.704814+0000","last_peered":"2024-09-16T15:07:21.704814+0000","last_clean":"2024-09-16T15:07:21.704814+0000","last_became_active":"2024-09-16T15:07:11.062607+0000","last_became_peered":"2024-09-16T15:07:11.062607+0000","last_unstale":"2024-09-16T15:07:21.704814+0000","last_undegraded":"2024-09-16T15:07:21.704814+0000","last_fullsized":"2024-09-16T15:07:21.704814+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:09.570910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:09.570910+0000","last_clean_scrub_stamp":"2024-09-16T15:07:09.570910+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:23.466827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.707553+0000","last_change":"2024-09-16T15:07:17.669610+0000","last_active":"2024-09-16T15:07:21.707553+0000","last_peered":"2024-09-16T15:07:21.707553+0000","last_clean":"2024-09-16T15:07:21.707553+0000","last_became_active":"2024-09-16T15:07:15.657618+0000","last_became_peered":"2024-09-16T15:07:15.657618+0000","last_unstale":"2024-09-16T15:07:21.707553+0000","last_undegraded":"2024-09-16T15:07:21.707553+0000","last_fullsized":"2024-09-16T15:07:21.707553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:28:19.381043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085582900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067153+0000","last_change":"2024-09-16T15:07:21.741970+0000","last_active":"2024-09-16T15:07:22.067153+0000","last_peered":"2024-09-16T15:07:22.067153+0000","last_clean":"2024-09-16T15:07:22.067153+0000","last_became_active":"2024-09-16T15:07:21.741305+0000","last_became_peered":"2024-09-16T15:07:21.741305+0000","last_unstale":"2024-09-16T15:07:22.067153+0000","last_undegraded":"2024-09-16T15:07:22.067153+0000","last_fullsized":"2024-09-16T15:07:22.067153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:46.566147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.096998+0000","last_change":"2024-09-16T15:07:22.095031+0000","last_active":"2024-09-16T15:07:22.096998+0000","last_peered":"2024-09-16T15:07:22.096998+0000","last_clean":"2024-09-16T15:07:22.096998+0000","last_became_active":"2024-09-16T15:07:22.094793+0000","last_became_peered":"2024-09-16T15:07:22.094793+0000","last_unstale":"2024-09-16T15:07:22.096998+0000","last_undegraded":"2024-09-16T15:07:22.096998+0000","last_fullsized":"2024-09-16T15:07:22.096998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:38:36.377299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.703762+0000","last_change":"2024-09-16T15:07:17.669433+0000","last_active":"2024-09-16T15:07:21.703762+0000","last_peered":"2024-09-16T15:07:21.703762+0000","last_clean":"2024-09-16T15:07:21.703762+0000","last_became_active":"2024-09-16T15:07:15.656237+0000","last_became_peered":"2024-09-16T15:07:15.656237+0000","last_unstale":"2024-09-16T15:07:21.703762+0000","last_undegraded":"2024-09-16T15:07:21.703762+0000","last_fullsized":"2024-09-16T15:07:21.703762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:49.124474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068265100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.055637+0000","last_change":"2024-09-16T15:07:21.713506+0000","last_active":"2024-09-16T15:07:22.055637+0000","last_peered":"2024-09-16T15:07:22.055637+0000","last_clean":"2024-09-16T15:07:22.055637+0000","last_became_active":"2024-09-16T15:07:21.713226+0000","last_became_peered":"2024-09-16T15:07:21.713226+0000","last_unstale":"2024-09-16T15:07:22.055637+0000","last_undegraded":"2024-09-16T15:07:22.055637+0000","last_fullsized":"2024-09-16T15:07:22.055637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082923+0000","last_change":"2024-09-16T15:07:22.079061+0000","last_active":"2024-09-16T15:07:22.082923+0000","last_peered":"2024-09-16T15:07:22.082923+0000","last_clean":"2024-09-16T15:07:22.082923+0000","last_became_active":"2024-09-16T15:07:22.076697+0000","last_became_peered":"2024-09-16T15:07:22.076697+0000","last_unstale":"2024-09-16T15:07:22.082923+0000","last_undegraded":"2024-09-16T15:07:22.082923+0000","last_fullsized":"2024-09-16T15:07:22.082923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:58.996879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.718834+0000","last_change":"2024-09-16T15:07:18.173036+0000","last_active":"2024-09-16T15:07:21.718834+0000","last_peered":"2024-09-16T15:07:21.718834+0000","last_clean":"2024-09-16T15:07:21.718834+0000","last_became_active":"2024-09-16T15:07:15.656238+0000","last_became_peered":"2024-09-16T15:07:15.656238+0000","last_unstale":"2024-09-16T15:07:21.718834+0000","last_undegraded":"2024-09-16T15:07:21.718834+0000","last_fullsized":"2024-09-16T15:07:21.718834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:15:02.977857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080979399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.053789+0000","last_change":"2024-09-16T15:07:21.722652+0000","last_active":"2024-09-16T15:07:22.053789+0000","last_peered":"2024-09-16T15:07:22.053789+0000","last_clean":"2024-09-16T15:07:22.053789+0000","last_became_active":"2024-09-16T15:07:21.722308+0000","last_became_peered":"2024-09-16T15:07:21.722308+0000","last_unstale":"2024-09-16T15:07:22.053789+0000","last_undegraded":"2024-09-16T15:07:22.053789+0000","last_fullsized":"2024-09-16T15:07:22.053789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.088775+0000","last_change":"2024-09-16T15:07:22.087268+0000","last_active":"2024-09-16T15:07:22.088775+0000","last_peered":"2024-09-16T15:07:22.088775+0000","last_clean":"2024-09-16T15:07:22.088775+0000","last_became_active":"2024-09-16T15:07:22.086932+0000","last_became_peered":"2024-09-16T15:07:22.086932+0000","last_unstale":"2024-09-16T15:07:22.088775+0000","last_undegraded":"2024-09-16T15:07:22.088775+0000","last_fullsized":"2024-09-16T15:07:22.088775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:55.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.705483+0000","last_change":"2024-09-16T15:07:17.671010+0000","last_active":"2024-09-16T15:07:21.705483+0000","last_peered":"2024-09-16T15:07:21.705483+0000","last_clean":"2024-09-16T15:07:21.705483+0000","last_became_active":"2024-09-16T15:07:15.658625+0000","last_became_peered":"2024-09-16T15:07:15.658625+0000","last_unstale":"2024-09-16T15:07:21.705483+0000","last_undegraded":"2024-09-16T15:07:21.705483+0000","last_fullsized":"2024-09-16T15:07:21.705483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:18.695305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00224435,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.055332+0000","last_change":"2024-09-16T15:07:21.712013+0000","last_active":"2024-09-16T15:07:22.055332+0000","last_peered":"2024-09-16T15:07:22.055332+0000","last_clean":"2024-09-16T15:07:22.055332+0000","last_became_active":"2024-09-16T15:07:21.710551+0000","last_became_peered":"2024-09-16T15:07:21.710551+0000","last_unstale":"2024-09-16T15:07:22.055332+0000","last_undegraded":"2024-09-16T15:07:22.055332+0000","last_fullsized":"2024-09-16T15:07:22.055332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.069566+0000","last_change":"2024-09-16T15:07:22.067593+0000","last_active":"2024-09-16T15:07:22.069566+0000","last_peered":"2024-09-16T15:07:22.069566+0000","last_clean":"2024-09-16T15:07:22.069566+0000","last_became_active":"2024-09-16T15:07:22.067254+0000","last_became_peered":"2024-09-16T15:07:22.067254+0000","last_unstale":"2024-09-16T15:07:22.069566+0000","last_undegraded":"2024-09-16T15:07:22.069566+0000","last_fullsized":"2024-09-16T15:07:22.069566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:43.560388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.054647+0000","last_change":"2024-09-16T15:07:21.711346+0000","last_active":"2024-09-16T15:07:22.054647+0000","last_peered":"2024-09-16T15:07:22.054647+0000","last_clean":"2024-09-16T15:07:22.054647+0000","last_became_active":"2024-09-16T15:07:21.710693+0000","last_became_peered":"2024-09-16T15:07:21.710693+0000","last_unstale":"2024-09-16T15:07:22.054647+0000","last_undegraded":"2024-09-16T15:07:22.054647+0000","last_fullsized":"2024-09-16T15:07:22.054647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:50.814161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.685865+0000","last_change":"2024-09-16T15:07:17.664832+0000","last_active":"2024-09-16T15:07:21.685865+0000","last_peered":"2024-09-16T15:07:21.685865+0000","last_clean":"2024-09-16T15:07:21.685865+0000","last_became_active":"2024-09-16T15:07:15.654955+0000","last_became_peered":"2024-09-16T15:07:15.654955+0000","last_unstale":"2024-09-16T15:07:21.685865+0000","last_undegraded":"2024-09-16T15:07:21.685865+0000","last_fullsized":"2024-09-16T15:07:21.685865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:48:38.399733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085315100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097557+0000","last_change":"2024-09-16T15:07:22.091087+0000","last_active":"2024-09-16T15:07:22.097557+0000","last_peered":"2024-09-16T15:07:22.097557+0000","last_clean":"2024-09-16T15:07:22.097557+0000","last_became_active":"2024-09-16T15:07:22.088585+0000","last_became_peered":"2024-09-16T15:07:22.088585+0000","last_unstale":"2024-09-16T15:07:22.097557+0000","last_undegraded":"2024-09-16T15:07:22.097557+0000","last_fullsized":"2024-09-16T15:07:22.097557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:19.377720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.055132+0000","last_change":"2024-09-16T15:07:21.713440+0000","last_active":"2024-09-16T15:07:22.055132+0000","last_peered":"2024-09-16T15:07:22.055132+0000","last_clean":"2024-09-16T15:07:22.055132+0000","last_became_active":"2024-09-16T15:07:21.713127+0000","last_became_peered":"2024-09-16T15:07:21.713127+0000","last_unstale":"2024-09-16T15:07:22.055132+0000","last_undegraded":"2024-09-16T15:07:22.055132+0000","last_fullsized":"2024-09-16T15:07:22.055132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.714217+0000","last_change":"2024-09-16T15:07:17.671471+0000","last_active":"2024-09-16T15:07:21.714217+0000","last_peered":"2024-09-16T15:07:21.714217+0000","last_clean":"2024-09-16T15:07:21.714217+0000","last_became_active":"2024-09-16T15:07:15.656288+0000","last_became_peered":"2024-09-16T15:07:15.656288+0000","last_unstale":"2024-09-16T15:07:21.714217+0000","last_undegraded":"2024-09-16T15:07:21.714217+0000","last_fullsized":"2024-09-16T15:07:21.714217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:03.700310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026747759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.096370+0000","last_change":"2024-09-16T15:07:22.087230+0000","last_active":"2024-09-16T15:07:22.096370+0000","last_peered":"2024-09-16T15:07:22.096370+0000","last_clean":"2024-09-16T15:07:22.096370+0000","last_became_active":"2024-09-16T15:07:22.086917+0000","last_became_peered":"2024-09-16T15:07:22.086917+0000","last_unstale":"2024-09-16T15:07:22.096370+0000","last_undegraded":"2024-09-16T15:07:22.096370+0000","last_fullsized":"2024-09-16T15:07:22.096370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:05.650468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.077952+0000","last_change":"2024-09-16T15:07:21.727506+0000","last_active":"2024-09-16T15:07:22.077952+0000","last_peered":"2024-09-16T15:07:22.077952+0000","last_clean":"2024-09-16T15:07:22.077952+0000","last_became_active":"2024-09-16T15:07:21.724965+0000","last_became_peered":"2024-09-16T15:07:21.724965+0000","last_unstale":"2024-09-16T15:07:22.077952+0000","last_undegraded":"2024-09-16T15:07:22.077952+0000","last_fullsized":"2024-09-16T15:07:22.077952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:41.835469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.686016+0000","last_change":"2024-09-16T15:07:17.664973+0000","last_active":"2024-09-16T15:07:21.686016+0000","last_peered":"2024-09-16T15:07:21.686016+0000","last_clean":"2024-09-16T15:07:21.686016+0000","last_became_active":"2024-09-16T15:07:15.655047+0000","last_became_peered":"2024-09-16T15:07:15.655047+0000","last_unstale":"2024-09-16T15:07:21.686016+0000","last_undegraded":"2024-09-16T15:07:21.686016+0000","last_fullsized":"2024-09-16T15:07:21.686016+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:53.308490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081824399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.103422+0000","last_change":"2024-09-16T15:07:22.100811+0000","last_active":"2024-09-16T15:07:22.103422+0000","last_peered":"2024-09-16T15:07:22.103422+0000","last_clean":"2024-09-16T15:07:22.103422+0000","last_became_active":"2024-09-16T15:07:22.099732+0000","last_became_peered":"2024-09-16T15:07:22.099732+0000","last_unstale":"2024-09-16T15:07:22.103422+0000","last_undegraded":"2024-09-16T15:07:22.103422+0000","last_fullsized":"2024-09-16T15:07:22.103422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:36.011335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.096463+0000","last_change":"2024-09-16T15:07:21.733464+0000","last_active":"2024-09-16T15:07:22.096463+0000","last_peered":"2024-09-16T15:07:22.096463+0000","last_clean":"2024-09-16T15:07:22.096463+0000","last_became_active":"2024-09-16T15:07:21.732659+0000","last_became_peered":"2024-09-16T15:07:21.732659+0000","last_unstale":"2024-09-16T15:07:22.096463+0000","last_undegraded":"2024-09-16T15:07:22.096463+0000","last_fullsized":"2024-09-16T15:07:22.096463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:07:21.704487+0000","last_change":"2024-09-16T15:07:17.671161+0000","last_active":"2024-09-16T15:07:21.704487+0000","last_peered":"2024-09-16T15:07:21.704487+0000","last_clean":"2024-09-16T15:07:21.704487+0000","last_became_active":"2024-09-16T15:07:15.654237+0000","last_became_peered":"2024-09-16T15:07:15.654237+0000","last_unstale":"2024-09-16T15:07:21.704487+0000","last_undegraded":"2024-09-16T15:07:21.704487+0000","last_fullsized":"2024-09-16T15:07:21.704487+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:56.780961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356239,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.075216+0000","last_change":"2024-09-16T15:07:22.071750+0000","last_active":"2024-09-16T15:07:22.075216+0000","last_peered":"2024-09-16T15:07:22.075216+0000","last_clean":"2024-09-16T15:07:22.075216+0000","last_became_active":"2024-09-16T15:07:22.071469+0000","last_became_peered":"2024-09-16T15:07:22.071469+0000","last_unstale":"2024-09-16T15:07:22.075216+0000","last_undegraded":"2024-09-16T15:07:22.075216+0000","last_fullsized":"2024-09-16T15:07:22.075216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:45.906789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.084182+0000","last_change":"2024-09-16T15:07:22.080700+0000","last_active":"2024-09-16T15:07:22.084182+0000","last_peered":"2024-09-16T15:07:22.084182+0000","last_clean":"2024-09-16T15:07:22.084182+0000","last_became_active":"2024-09-16T15:07:22.080300+0000","last_became_peered":"2024-09-16T15:07:22.080300+0000","last_unstale":"2024-09-16T15:07:22.084182+0000","last_undegraded":"2024-09-16T15:07:22.084182+0000","last_fullsized":"2024-09-16T15:07:22.084182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:27.703305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.096828+0000","last_change":"2024-09-16T15:07:21.741959+0000","last_active":"2024-09-16T15:07:22.096828+0000","last_peered":"2024-09-16T15:07:22.096828+0000","last_clean":"2024-09-16T15:07:22.096828+0000","last_became_active":"2024-09-16T15:07:21.741329+0000","last_became_peered":"2024-09-16T15:07:21.741329+0000","last_unstale":"2024-09-16T15:07:22.096828+0000","last_undegraded":"2024-09-16T15:07:22.096828+0000","last_fullsized":"2024-09-16T15:07:22.096828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:31.395563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.097512+0000","last_change":"2024-09-16T15:07:22.088939+0000","last_active":"2024-09-16T15:07:22.097512+0000","last_peered":"2024-09-16T15:07:22.097512+0000","last_clean":"2024-09-16T15:07:22.097512+0000","last_became_active":"2024-09-16T15:07:22.087096+0000","last_became_peered":"2024-09-16T15:07:22.087096+0000","last_unstale":"2024-09-16T15:07:22.097512+0000","last_undegraded":"2024-09-16T15:07:22.097512+0000","last_fullsized":"2024-09-16T15:07:22.097512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:49.184359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.080521+0000","last_change":"2024-09-16T15:07:21.737806+0000","last_active":"2024-09-16T15:07:22.080521+0000","last_peered":"2024-09-16T15:07:22.080521+0000","last_clean":"2024-09-16T15:07:22.080521+0000","last_became_active":"2024-09-16T15:07:21.737289+0000","last_became_peered":"2024-09-16T15:07:21.737289+0000","last_unstale":"2024-09-16T15:07:22.080521+0000","last_undegraded":"2024-09-16T15:07:22.080521+0000","last_fullsized":"2024-09-16T15:07:22.080521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:15.691812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.055771+0000","last_change":"2024-09-16T15:07:21.738040+0000","last_active":"2024-09-16T15:07:22.055771+0000","last_peered":"2024-09-16T15:07:22.055771+0000","last_clean":"2024-09-16T15:07:22.055771+0000","last_became_active":"2024-09-16T15:07:21.737276+0000","last_became_peered":"2024-09-16T15:07:21.737276+0000","last_unstale":"2024-09-16T15:07:22.055771+0000","last_undegraded":"2024-09-16T15:07:22.055771+0000","last_fullsized":"2024-09-16T15:07:22.055771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.074708+0000","last_change":"2024-09-16T15:07:22.073415+0000","last_active":"2024-09-16T15:07:22.074708+0000","last_peered":"2024-09-16T15:07:22.074708+0000","last_clean":"2024-09-16T15:07:22.074708+0000","last_became_active":"2024-09-16T15:07:22.073160+0000","last_became_peered":"2024-09-16T15:07:22.073160+0000","last_unstale":"2024-09-16T15:07:22.074708+0000","last_undegraded":"2024-09-16T15:07:22.074708+0000","last_fullsized":"2024-09-16T15:07:22.074708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:32.398564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.095864+0000","last_change":"2024-09-16T15:07:21.730081+0000","last_active":"2024-09-16T15:07:22.095864+0000","last_peered":"2024-09-16T15:07:22.095864+0000","last_clean":"2024-09-16T15:07:22.095864+0000","last_became_active":"2024-09-16T15:07:21.729450+0000","last_became_peered":"2024-09-16T15:07:21.729450+0000","last_unstale":"2024-09-16T15:07:22.095864+0000","last_undegraded":"2024-09-16T15:07:22.095864+0000","last_fullsized":"2024-09-16T15:07:22.095864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.101762+0000","last_change":"2024-09-16T15:07:22.100315+0000","last_active":"2024-09-16T15:07:22.101762+0000","last_peered":"2024-09-16T15:07:22.101762+0000","last_clean":"2024-09-16T15:07:22.101762+0000","last_became_active":"2024-09-16T15:07:22.099367+0000","last_became_peered":"2024-09-16T15:07:22.099367+0000","last_unstale":"2024-09-16T15:07:22.101762+0000","last_undegraded":"2024-09-16T15:07:22.101762+0000","last_fullsized":"2024-09-16T15:07:22.101762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:45.128492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.082965+0000","last_change":"2024-09-16T15:07:22.079005+0000","last_active":"2024-09-16T15:07:22.082965+0000","last_peered":"2024-09-16T15:07:22.082965+0000","last_clean":"2024-09-16T15:07:22.082965+0000","last_became_active":"2024-09-16T15:07:22.077888+0000","last_became_peered":"2024-09-16T15:07:22.077888+0000","last_unstale":"2024-09-16T15:07:22.082965+0000","last_undegraded":"2024-09-16T15:07:22.082965+0000","last_fullsized":"2024-09-16T15:07:22.082965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:21.103662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.064628+0000","last_change":"2024-09-16T15:07:21.724826+0000","last_active":"2024-09-16T15:07:22.064628+0000","last_peered":"2024-09-16T15:07:22.064628+0000","last_clean":"2024-09-16T15:07:22.064628+0000","last_became_active":"2024-09-16T15:07:21.724152+0000","last_became_peered":"2024-09-16T15:07:21.724152+0000","last_unstale":"2024-09-16T15:07:22.064628+0000","last_undegraded":"2024-09-16T15:07:22.064628+0000","last_fullsized":"2024-09-16T15:07:22.064628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:26.700305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.088717+0000","last_change":"2024-09-16T15:07:22.087018+0000","last_active":"2024-09-16T15:07:22.088717+0000","last_peered":"2024-09-16T15:07:22.088717+0000","last_clean":"2024-09-16T15:07:22.088717+0000","last_became_active":"2024-09-16T15:07:22.086746+0000","last_became_peered":"2024-09-16T15:07:22.086746+0000","last_unstale":"2024-09-16T15:07:22.088717+0000","last_undegraded":"2024-09-16T15:07:22.088717+0000","last_fullsized":"2024-09-16T15:07:22.088717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:11.167685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.093649+0000","last_change":"2024-09-16T15:07:21.726511+0000","last_active":"2024-09-16T15:07:22.093649+0000","last_peered":"2024-09-16T15:07:22.093649+0000","last_clean":"2024-09-16T15:07:22.093649+0000","last_became_active":"2024-09-16T15:07:21.725971+0000","last_became_peered":"2024-09-16T15:07:21.725971+0000","last_unstale":"2024-09-16T15:07:22.093649+0000","last_undegraded":"2024-09-16T15:07:22.093649+0000","last_fullsized":"2024-09-16T15:07:22.093649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:15.691812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.095299+0000","last_change":"2024-09-16T15:07:22.093131+0000","last_active":"2024-09-16T15:07:22.095299+0000","last_peered":"2024-09-16T15:07:22.095299+0000","last_clean":"2024-09-16T15:07:22.095299+0000","last_became_active":"2024-09-16T15:07:22.092810+0000","last_became_peered":"2024-09-16T15:07:22.092810+0000","last_unstale":"2024-09-16T15:07:22.095299+0000","last_undegraded":"2024-09-16T15:07:22.095299+0000","last_fullsized":"2024-09-16T15:07:22.095299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:27.872651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.100731+0000","last_change":"2024-09-16T15:07:21.709700+0000","last_active":"2024-09-16T15:07:22.100731+0000","last_peered":"2024-09-16T15:07:22.100731+0000","last_clean":"2024-09-16T15:07:22.100731+0000","last_became_active":"2024-09-16T15:07:21.708187+0000","last_became_peered":"2024-09-16T15:07:21.708187+0000","last_unstale":"2024-09-16T15:07:22.100731+0000","last_undegraded":"2024-09-16T15:07:22.100731+0000","last_fullsized":"2024-09-16T15:07:22.100731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:50.814161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.069607+0000","last_change":"2024-09-16T15:07:22.067775+0000","last_active":"2024-09-16T15:07:22.069607+0000","last_peered":"2024-09-16T15:07:22.069607+0000","last_clean":"2024-09-16T15:07:22.069607+0000","last_became_active":"2024-09-16T15:07:22.067259+0000","last_became_peered":"2024-09-16T15:07:22.067259+0000","last_unstale":"2024-09-16T15:07:22.069607+0000","last_undegraded":"2024-09-16T15:07:22.069607+0000","last_fullsized":"2024-09-16T15:07:22.069607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:04.412982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.092710+0000","last_change":"2024-09-16T15:07:21.726917+0000","last_active":"2024-09-16T15:07:22.092710+0000","last_peered":"2024-09-16T15:07:22.092710+0000","last_clean":"2024-09-16T15:07:22.092710+0000","last_became_active":"2024-09-16T15:07:21.724806+0000","last_became_peered":"2024-09-16T15:07:21.724806+0000","last_unstale":"2024-09-16T15:07:22.092710+0000","last_undegraded":"2024-09-16T15:07:22.092710+0000","last_fullsized":"2024-09-16T15:07:22.092710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:54.201635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067214+0000","last_change":"2024-09-16T15:07:22.059433+0000","last_active":"2024-09-16T15:07:22.067214+0000","last_peered":"2024-09-16T15:07:22.067214+0000","last_clean":"2024-09-16T15:07:22.067214+0000","last_became_active":"2024-09-16T15:07:22.058973+0000","last_became_peered":"2024-09-16T15:07:22.058973+0000","last_unstale":"2024-09-16T15:07:22.067214+0000","last_undegraded":"2024-09-16T15:07:22.067214+0000","last_fullsized":"2024-09-16T15:07:22.067214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:11.167685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.101717+0000","last_change":"2024-09-16T15:07:21.726761+0000","last_active":"2024-09-16T15:07:22.101717+0000","last_peered":"2024-09-16T15:07:22.101717+0000","last_clean":"2024-09-16T15:07:22.101717+0000","last_became_active":"2024-09-16T15:07:21.726186+0000","last_became_peered":"2024-09-16T15:07:21.726186+0000","last_unstale":"2024-09-16T15:07:22.101717+0000","last_undegraded":"2024-09-16T15:07:22.101717+0000","last_fullsized":"2024-09-16T15:07:22.101717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.066230+0000","last_change":"2024-09-16T15:07:22.059781+0000","last_active":"2024-09-16T15:07:22.066230+0000","last_peered":"2024-09-16T15:07:22.066230+0000","last_clean":"2024-09-16T15:07:22.066230+0000","last_became_active":"2024-09-16T15:07:22.059423+0000","last_became_peered":"2024-09-16T15:07:22.059423+0000","last_unstale":"2024-09-16T15:07:22.066230+0000","last_undegraded":"2024-09-16T15:07:22.066230+0000","last_fullsized":"2024-09-16T15:07:22.066230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:22.080488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.078150+0000","last_change":"2024-09-16T15:07:21.743147+0000","last_active":"2024-09-16T15:07:22.078150+0000","last_peered":"2024-09-16T15:07:22.078150+0000","last_clean":"2024-09-16T15:07:22.078150+0000","last_became_active":"2024-09-16T15:07:21.742365+0000","last_became_peered":"2024-09-16T15:07:21.742365+0000","last_unstale":"2024-09-16T15:07:22.078150+0000","last_undegraded":"2024-09-16T15:07:22.078150+0000","last_fullsized":"2024-09-16T15:07:22.078150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:10.164684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.083121+0000","last_change":"2024-09-16T15:07:22.076343+0000","last_active":"2024-09-16T15:07:22.083121+0000","last_peered":"2024-09-16T15:07:22.083121+0000","last_clean":"2024-09-16T15:07:22.083121+0000","last_became_active":"2024-09-16T15:07:22.075982+0000","last_became_peered":"2024-09-16T15:07:22.075982+0000","last_unstale":"2024-09-16T15:07:22.083121+0000","last_undegraded":"2024-09-16T15:07:22.083121+0000","last_fullsized":"2024-09-16T15:07:22.083121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:25:32.065204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.067836+0000","last_change":"2024-09-16T15:07:21.722589+0000","last_active":"2024-09-16T15:07:22.067836+0000","last_peered":"2024-09-16T15:07:22.067836+0000","last_clean":"2024-09-16T15:07:22.067836+0000","last_became_active":"2024-09-16T15:07:21.722226+0000","last_became_peered":"2024-09-16T15:07:21.722226+0000","last_unstale":"2024-09-16T15:07:22.067836+0000","last_undegraded":"2024-09-16T15:07:22.067836+0000","last_fullsized":"2024-09-16T15:07:22.067836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.066275+0000","last_change":"2024-09-16T15:07:22.061731+0000","last_active":"2024-09-16T15:07:22.066275+0000","last_peered":"2024-09-16T15:07:22.066275+0000","last_clean":"2024-09-16T15:07:22.066275+0000","last_became_active":"2024-09-16T15:07:22.061419+0000","last_became_peered":"2024-09-16T15:07:22.061419+0000","last_unstale":"2024-09-16T15:07:22.066275+0000","last_undegraded":"2024-09-16T15:07:22.066275+0000","last_fullsized":"2024-09-16T15:07:22.066275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:47.569148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.103592+0000","last_change":"2024-09-16T15:07:21.709152+0000","last_active":"2024-09-16T15:07:22.103592+0000","last_peered":"2024-09-16T15:07:22.103592+0000","last_clean":"2024-09-16T15:07:22.103592+0000","last_became_active":"2024-09-16T15:07:21.708468+0000","last_became_peered":"2024-09-16T15:07:21.708468+0000","last_unstale":"2024-09-16T15:07:22.103592+0000","last_undegraded":"2024-09-16T15:07:22.103592+0000","last_fullsized":"2024-09-16T15:07:22.103592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":64588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":794624,"data_stored":653997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":647783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":172032,"data_stored":57434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:07:26.082 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-16T15:07:26.083 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T15:07:26.083 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T15:07:26.083 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T15:07:26.083 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T15:07:26.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.238+0000 7fd79be3d640 1 Processor -- start 2024-09-16T15:07:26.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.238+0000 7fd79be3d640 1 -- start start 2024-09-16T15:07:26.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.238+0000 7fd79be3d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7941365b0 0x7fd7940aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.238+0000 7fd79be3d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7940a9ae0 0x7fd7940a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.238+0000 7fd79be3d640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd7940ab810 0x7fd7940a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.238+0000 7fd79be3d640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7941399f0 con 0x7fd7941365b0 2024-09-16T15:07:26.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.238+0000 7fd79be3d640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd794139b60 con 0x7fd7940ab810 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.238+0000 7fd79be3d640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940a5ca0 con 0x7fd7940a9ae0 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd799bd9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7941365b0 0x7fd7940aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd7993d8640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7940a9ae0 0x7fd7940a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd799bd9640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd7941365b0 0x7fd7940aa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51710/0 (socket says 172.21.15.133:51710) 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd799bd9640 1 -- 172.21.15.133:0/2212376699 learned_addr learned my addr 172.21.15.133:0/2212376699 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd79a3da640 1 --2- 172.21.15.133:0/2212376699 >> v2:172.21.15.191:3300/0 conn(0x7fd7940ab810 0x7fd7940a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd799bd9640 1 -- 172.21.15.133:0/2212376699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7940a9ae0 msgr2=0x7fd7940a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd799bd9640 1 --2- 172.21.15.133:0/2212376699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7940a9ae0 0x7fd7940a80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd799bd9640 1 -- 172.21.15.133:0/2212376699 >> v2:172.21.15.191:3300/0 conn(0x7fd7940ab810 msgr2=0x7fd7940a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd799bd9640 1 --2- 172.21.15.133:0/2212376699 >> v2:172.21.15.191:3300/0 conn(0x7fd7940ab810 0x7fd7940a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd799bd9640 1 -- 172.21.15.133:0/2212376699 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd794136990 con 0x7fd7941365b0 2024-09-16T15:07:26.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd7993d8640 1 --2- 172.21.15.133:0/2212376699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7940a9ae0 0x7fd7940a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:26.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd799bd9640 1 --2- 172.21.15.133:0/2212376699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7941365b0 0x7fd7940aa960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd794147990 tx=0x7fd79001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba40c6bafc79bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:26.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd798bd7640 1 -- 172.21.15.133:0/2212376699 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7900ab600 con 0x7fd7941365b0 2024-09-16T15:07:26.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd798bd7640 1 -- 172.21.15.133:0/2212376699 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7900a7050 con 0x7fd7941365b0 2024-09-16T15:07:26.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.242+0000 7fd798bd7640 1 -- 172.21.15.133:0/2212376699 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7900ab980 con 0x7fd7941365b0 2024-09-16T15:07:26.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- 172.21.15.133:0/2212376699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7941365b0 msgr2=0x7fd7940aa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 --2- 172.21.15.133:0/2212376699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7941365b0 0x7fd7940aa960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd794147990 tx=0x7fd79001c3e0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- 172.21.15.133:0/2212376699 shutdown_connections 2024-09-16T15:07:26.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 --2- 172.21.15.133:0/2212376699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7940a9ae0 0x7fd7940a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 --2- 172.21.15.133:0/2212376699 >> v2:172.21.15.191:3300/0 conn(0x7fd7940ab810 0x7fd7940a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 --2- 172.21.15.133:0/2212376699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7941365b0 0x7fd7940aa960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- 172.21.15.133:0/2212376699 >> 172.21.15.133:0/2212376699 conn(0x7fd79406a030 msgr2=0x7fd79406bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:26.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- 172.21.15.133:0/2212376699 shutdown_connections 2024-09-16T15:07:26.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- 172.21.15.133:0/2212376699 wait complete. 2024-09-16T15:07:26.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 Processor -- start 2024-09-16T15:07:26.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- start start 2024-09-16T15:07:26.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7940a9ae0 0x7fd7941db2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7940ab810 0x7fd794145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7941365b0 0x7fd794145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79414caf0 con 0x7fd7940ab810 2024-09-16T15:07:26.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd794148790 con 0x7fd7940a9ae0 2024-09-16T15:07:26.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941dbc00 con 0x7fd7941365b0 2024-09-16T15:07:26.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd7993d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7940ab810 0x7fd794145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79a3da640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7941365b0 0x7fd794145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd7993d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7940ab810 0x7fd794145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51722/0 (socket says 172.21.15.133:51722) 2024-09-16T15:07:26.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd7993d8640 1 -- 172.21.15.133:0/2476092673 learned_addr learned my addr 172.21.15.133:0/2476092673 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:26.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd799bd9640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7940a9ae0 0x7fd7941db2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd7993d8640 1 -- 172.21.15.133:0/2476092673 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7941365b0 msgr2=0x7fd794145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd7993d8640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7941365b0 0x7fd794145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd7993d8640 1 -- 172.21.15.133:0/2476092673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7940a9ae0 msgr2=0x7fd7941db2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd7993d8640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7940a9ae0 0x7fd7941db2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd7993d8640 1 -- 172.21.15.133:0/2476092673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd784009420 con 0x7fd7940ab810 2024-09-16T15:07:26.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd7993d8640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7940ab810 0x7fd794145750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd78400ea00 tx=0x7fd78400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:26.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd78a7fc640 1 -- 172.21.15.133:0/2476092673 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78400ca60 con 0x7fd7940ab810 2024-09-16T15:07:26.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd78a7fc640 1 -- 172.21.15.133:0/2476092673 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78400cbc0 con 0x7fd7940ab810 2024-09-16T15:07:26.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- 172.21.15.133:0/2476092673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7900094d0 con 0x7fd7940ab810 2024-09-16T15:07:26.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd78a7fc640 1 -- 172.21.15.133:0/2476092673 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784013470 con 0x7fd7940ab810 2024-09-16T15:07:26.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.246+0000 7fd79be3d640 1 -- 172.21.15.133:0/2476092673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7940a58b0 con 0x7fd7940ab810 2024-09-16T15:07:26.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.250+0000 7fd79be3d640 1 -- 172.21.15.133:0/2476092673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7940672a0 con 0x7fd7940ab810 2024-09-16T15:07:26.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.250+0000 7fd78a7fc640 1 -- 172.21.15.133:0/2476092673 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fd7840135d0 con 0x7fd7940ab810 2024-09-16T15:07:26.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.250+0000 7fd78a7fc640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:26.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.250+0000 7fd78a7fc640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:26.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.250+0000 7fd78a7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:26.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.250+0000 7fd78a7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:26.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.250+0000 7fd78a7fc640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd7780978c0 0x7fd778099d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.250+0000 7fd78a7fc640 1 -- 172.21.15.133:0/2476092673 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4362+0+0 (secure 0 0 0) 0x7fd7840bfcd0 con 0x7fd7940ab810 2024-09-16T15:07:26.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.250+0000 7fd799bd9640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd7780978c0 0x7fd778099d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.250+0000 7fd799bd9640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd7780978c0 0x7fd778099d80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7900ad000 tx=0x7fd790008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:26.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.254+0000 7fd78a7fc640 1 -- 172.21.15.133:0/2476092673 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd7840c5020 con 0x7fd7940ab810 2024-09-16T15:07:26.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.442+0000 7fd79be3d640 1 -- 172.21.15.133:0/2476092673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd7940b9ff0 con 0x7fd7940ab810 2024-09-16T15:07:26.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.454+0000 7fd78a7fc640 1 -- 172.21.15.133:0/2476092673 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+246693 (secure 0 0 0) 0x7fd78408d340 con 0x7fd7940ab810 2024-09-16T15:07:26.460 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:26.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 10 mgrc shutdown 2024-09-16T15:07:26.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 -- 172.21.15.133:0/2476092673 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd7780978c0 msgr2=0x7fd778099d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd7780978c0 0x7fd778099d80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7900ad000 tx=0x7fd790008040 comp rx=0 tx=0).stop 2024-09-16T15:07:26.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 -- 172.21.15.133:0/2476092673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7940ab810 msgr2=0x7fd794145750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7940ab810 0x7fd794145750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd78400ea00 tx=0x7fd78400eec0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 -- 172.21.15.133:0/2476092673 shutdown_connections 2024-09-16T15:07:26.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7941365b0 0x7fd794145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd7780978c0 0x7fd778099d80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7940ab810 0x7fd794145750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 --2- 172.21.15.133:0/2476092673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7940a9ae0 0x7fd7941db2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 -- 172.21.15.133:0/2476092673 >> 172.21.15.133:0/2476092673 conn(0x7fd79406a030 msgr2=0x7fd7940a80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:26.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 -- 172.21.15.133:0/2476092673 shutdown_connections 2024-09-16T15:07:26.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.458+0000 7fd79be3d640 1 -- 172.21.15.133:0/2476092673 wait complete. 2024-09-16T15:07:26.476 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"z","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3926506186},{"type":"v1","addr":"172.21.15.191:6825","nonce":3926506186}]},"active_addr":"172.21.15.191:6825/3926506186","active_change":"2024-09-16T15:07:02.678974+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4113,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":4114,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":1928163351}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":1049131429}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":2465333633}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.191:0","nonce":3684856439}]}]} 2024-09-16T15:07:26.481 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T15:07:26.481 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T15:07:26.481 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:26.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5c5ba640 1 Processor -- start 2024-09-16T15:07:26.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5c5ba640 1 -- start start 2024-09-16T15:07:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5c5ba640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c5406afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5c5ba640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4c540a1010 0x7f4c5414bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5c5ba640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5406a8f0 con 0x7f4c540a2cf0 2024-09-16T15:07:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5c5ba640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c540a5bd0 con 0x7f4c540a1010 2024-09-16T15:07:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5c5ba640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540a4cf0 con 0x7f4c5409e490 2024-09-16T15:07:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5a356640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4c540a2cf0 0x7f4c540a53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52566/0 (socket says 172.21.15.133:52566) 2024-09-16T15:07:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5a356640 1 -- 172.21.15.133:0/2822727503 learned_addr learned my addr 172.21.15.133:0/2822727503 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:26.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c59b55640 1 --2- 172.21.15.133:0/2822727503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c5406afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5ab57640 1 --2- 172.21.15.133:0/2822727503 >> v2:172.21.15.133:3300/0 conn(0x7f4c540a1010 0x7f4c5414bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c59b55640 1 -- 172.21.15.133:0/2822727503 >> v2:172.21.15.133:3300/0 conn(0x7f4c540a1010 msgr2=0x7f4c5414bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c59b55640 1 --2- 172.21.15.133:0/2822727503 >> v2:172.21.15.133:3300/0 conn(0x7f4c540a1010 0x7f4c5414bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c59b55640 1 -- 172.21.15.133:0/2822727503 >> v1:172.21.15.133:6789/0 conn(0x7f4c540a2cf0 legacy=0x7f4c540a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c59b55640 1 -- 172.21.15.133:0/2822727503 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540a3950 con 0x7f4c5409e490 2024-09-16T15:07:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c5ab57640 1 --2- 172.21.15.133:0/2822727503 >> v2:172.21.15.133:3300/0 conn(0x7f4c540a1010 0x7f4c5414bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:26.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.630+0000 7f4c59354640 1 -- 172.21.15.133:0/2822727503 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3484468416 0 0) 0x7f4c5406a8f0 con 0x7f4c540a2cf0 2024-09-16T15:07:26.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c59b55640 1 --2- 172.21.15.133:0/2822727503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c5406afe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4c500097c0 tx=0x7f4c5001c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ef750c421fbe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:26.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c59354640 1 -- 172.21.15.133:0/2822727503 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c500ad680 con 0x7f4c5409e490 2024-09-16T15:07:26.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c59354640 1 -- 172.21.15.133:0/2822727503 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c500ad7e0 con 0x7f4c5409e490 2024-09-16T15:07:26.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c59354640 1 -- 172.21.15.133:0/2822727503 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c500adb70 con 0x7f4c5409e490 2024-09-16T15:07:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/2822727503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 msgr2=0x7f4c5406afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c5c5ba640 1 --2- 172.21.15.133:0/2822727503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c5406afe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4c500097c0 tx=0x7f4c5001c360 comp rx=0 tx=0).stop 2024-09-16T15:07:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/2822727503 shutdown_connections 2024-09-16T15:07:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c5c5ba640 1 --2- 172.21.15.133:0/2822727503 >> v2:172.21.15.133:3300/0 conn(0x7f4c540a1010 0x7f4c5414bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c5c5ba640 1 --2- 172.21.15.133:0/2822727503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c5406afe0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/2822727503 >> 172.21.15.133:0/2822727503 conn(0x7f4c540a3fd0 msgr2=0x7f4c540a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/2822727503 shutdown_connections 2024-09-16T15:07:26.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/2822727503 wait complete. 2024-09-16T15:07:26.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c5c5ba640 1 Processor -- start 2024-09-16T15:07:26.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.634+0000 7f4c5c5ba640 1 -- start start 2024-09-16T15:07:26.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5c5ba640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c540b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5c5ba640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c540a1010 0x7f4c540713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5c5ba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c540a2cf0 0x7f4c5406f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5c5ba640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c541371c0 con 0x7f4c540a1010 2024-09-16T15:07:26.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5c5ba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54071900 con 0x7f4c540a2cf0 2024-09-16T15:07:26.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5c5ba640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54141510 con 0x7f4c5409e490 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5a356640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c540b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c59b55640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c540a1010 0x7f4c540713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5a356640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c540b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:34224/0 (socket says 172.21.15.133:34224) 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5a356640 1 -- 172.21.15.133:0/3950220346 learned_addr learned my addr 172.21.15.133:0/3950220346 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5ab57640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c540a2cf0 0x7f4c5406f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5a356640 1 -- 172.21.15.133:0/3950220346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c540a2cf0 msgr2=0x7f4c5406f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5a356640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c540a2cf0 0x7f4c5406f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5a356640 1 -- 172.21.15.133:0/3950220346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c540a1010 msgr2=0x7f4c540713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5a356640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c540a1010 0x7f4c540713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5a356640 1 -- 172.21.15.133:0/3950220346 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c009a70 con 0x7f4c5409e490 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c59b55640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c540a1010 0x7f4c540713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5a356640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c540b3650 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c520 tx=0x7f4c4400c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c4affd640 1 -- 172.21.15.133:0/3950220346 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440074e0 con 0x7f4c5409e490 2024-09-16T15:07:26.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c4affd640 1 -- 172.21.15.133:0/3950220346 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c44007640 con 0x7f4c5409e490 2024-09-16T15:07:26.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/3950220346 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c50009420 con 0x7f4c5409e490 2024-09-16T15:07:26.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c4affd640 1 -- 172.21.15.133:0/3950220346 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440049f0 con 0x7f4c5409e490 2024-09-16T15:07:26.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/3950220346 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c540709c0 con 0x7f4c5409e490 2024-09-16T15:07:26.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/3950220346 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c54137330 con 0x7f4c5409e490 2024-09-16T15:07:26.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c4affd640 1 -- 172.21.15.133:0/3950220346 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f4c4401b360 con 0x7f4c5409e490 2024-09-16T15:07:26.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c4affd640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:26.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c4affd640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:26.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c4affd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:26.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c4affd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:26.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c4affd640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4c38097840 0x7f4c38099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.638+0000 7f4c4affd640 1 -- 172.21.15.133:0/3950220346 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f4c440bf070 con 0x7f4c5409e490 2024-09-16T15:07:26.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.642+0000 7f4c59b55640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4c38097840 0x7f4c38099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.642+0000 7f4c59b55640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4c38097840 0x7f4c38099d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4c500af000 tx=0x7f4c500a7000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:26.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.642+0000 7f4c4affd640 1 -- 172.21.15.133:0/3950220346 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c4408c8c0 con 0x7f4c5409e490 2024-09-16T15:07:26.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.802+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/3950220346 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c54137330 con 0x7f4c5409e490 2024-09-16T15:07:26.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.802+0000 7f4c4affd640 1 -- 172.21.15.133:0/3950220346 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+13441 (secure 0 0 0) 0x7f4c440ac4a0 con 0x7f4c5409e490 2024-09-16T15:07:26.808 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:26.808 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:23.049976+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:07:20.523939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-16T15:07:21.043356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 10 mgrc shutdown 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/3950220346 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4c38097840 msgr2=0x7f4c38099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4c38097840 0x7f4c38099d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4c500af000 tx=0x7f4c500a7000 comp rx=0 tx=0).stop 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/3950220346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 msgr2=0x7f4c540b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c540b3650 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c520 tx=0x7f4c4400c9e0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/3950220346 shutdown_connections 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c540a2cf0 0x7f4c5406f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4c38097840 0x7f4c38099d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c540a1010 0x7f4c540713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 --2- 172.21.15.133:0/3950220346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5409e490 0x7f4c540b3650 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/3950220346 >> 172.21.15.133:0/3950220346 conn(0x7f4c540a3fd0 msgr2=0x7f4c54142f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:26.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/3950220346 shutdown_connections 2024-09-16T15:07:26.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.806+0000 7f4c5c5ba640 1 -- 172.21.15.133:0/3950220346 wait complete. 2024-09-16T15:07:26.825 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T15:07:26.825 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.926+0000 7f1fe1008640 1 Processor -- start 2024-09-16T15:07:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.926+0000 7f1fe1008640 1 -- start start 2024-09-16T15:07:26.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1fdc0a1010 0x7f1fdc14bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fdc06a8f0 con 0x7f1fdc09e490 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fdc0a5bd0 con 0x7f1fdc0a1010 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc0a5910 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fdad9d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fda59c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1fdc09e490 0x7f1fdc06afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52568/0 (socket says 172.21.15.133:52568) 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fda59c640 1 -- 172.21.15.133:0/2924414481 learned_addr learned my addr 172.21.15.133:0/2924414481 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fdb59e640 1 --2- 172.21.15.133:0/2924414481 >> v2:172.21.15.191:3300/0 conn(0x7f1fdc0a1010 0x7f1fdc14bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fdad9d640 1 -- 172.21.15.133:0/2924414481 >> v2:172.21.15.191:3300/0 conn(0x7f1fdc0a1010 msgr2=0x7f1fdc14bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fdad9d640 1 --2- 172.21.15.133:0/2924414481 >> v2:172.21.15.191:3300/0 conn(0x7f1fdc0a1010 0x7f1fdc14bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fdad9d640 1 -- 172.21.15.133:0/2924414481 >> v1:172.21.15.133:6789/0 conn(0x7f1fdc09e490 legacy=0x7f1fdc06afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fd9d9b640 1 -- 172.21.15.133:0/2924414481 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888237802 0 0) 0x7f1fdc06a8f0 con 0x7f1fdc09e490 2024-09-16T15:07:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fdad9d640 1 -- 172.21.15.133:0/2924414481 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc0a4cf0 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fdb59e640 1 --2- 172.21.15.133:0/2924414481 >> v2:172.21.15.191:3300/0 conn(0x7f1fdc0a1010 0x7f1fdc14bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fdad9d640 1 --2- 172.21.15.133:0/2924414481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc0a53d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0097c0 tx=0x7f1fcc01c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2179fb7f18f55559 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fd9d9b640 1 -- 172.21.15.133:0/2924414481 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc0ad660 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fd9d9b640 1 -- 172.21.15.133:0/2924414481 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc0ad7c0 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fd9d9b640 1 -- 172.21.15.133:0/2924414481 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc0adb50 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 -- 172.21.15.133:0/2924414481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 msgr2=0x7f1fdc0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 --2- 172.21.15.133:0/2924414481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc0a53d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0097c0 tx=0x7f1fcc01c360 comp rx=0 tx=0).stop 2024-09-16T15:07:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 -- 172.21.15.133:0/2924414481 shutdown_connections 2024-09-16T15:07:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 --2- 172.21.15.133:0/2924414481 >> v2:172.21.15.191:3300/0 conn(0x7f1fdc0a1010 0x7f1fdc14bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 --2- 172.21.15.133:0/2924414481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc0a53d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 -- 172.21.15.133:0/2924414481 >> 172.21.15.133:0/2924414481 conn(0x7f1fdc0a3fd0 msgr2=0x7f1fdc0a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 -- 172.21.15.133:0/2924414481 shutdown_connections 2024-09-16T15:07:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.930+0000 7f1fe1008640 1 -- 172.21.15.133:0/2924414481 wait complete. 2024-09-16T15:07:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 Processor -- start 2024-09-16T15:07:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 -- start start 2024-09-16T15:07:26.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fdc09e490 0x7f1fdc145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fdc0a1010 0x7f1fdc1415e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc1db4b0 con 0x7f1fdc09e490 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc140210 con 0x7f1fdc0a1010 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc141ec0 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdb59e640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc139aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fdc09e490 0x7f1fdc145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdb59e640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc139aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:34232/0 (socket says 172.21.15.133:34232) 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdb59e640 1 -- 172.21.15.133:0/1318971459 learned_addr learned my addr 172.21.15.133:0/1318971459 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fda59c640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fdc0a1010 0x7f1fdc1415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdb59e640 1 -- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fdc0a1010 msgr2=0x7f1fdc1415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdb59e640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fdc0a1010 0x7f1fdc1415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdb59e640 1 -- 172.21.15.133:0/1318971459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fdc09e490 msgr2=0x7f1fdc145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdb59e640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fdc09e490 0x7f1fdc145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdb59e640 1 -- 172.21.15.133:0/1318971459 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc009420 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fda59c640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fdc0a1010 0x7f1fdc1415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdad9d640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fdc09e490 0x7f1fdc145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fdb59e640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc139aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400cef0 tx=0x7f1fd400a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fd37fe640 1 -- 172.21.15.133:0/1318971459 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd400d840 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fd37fe640 1 -- 172.21.15.133:0/1318971459 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd400d9c0 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 -- 172.21.15.133:0/1318971459 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fdc06b870 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fd37fe640 1 -- 172.21.15.133:0/1318971459 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd400dd70 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 -- 172.21.15.133:0/1318971459 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fdc140e00 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.934+0000 7f1fe1008640 1 -- 172.21.15.133:0/1318971459 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fdc0a53d0 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.938+0000 7f1fd37fe640 1 -- 172.21.15.133:0/1318971459 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f1fd401e3b0 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.938+0000 7f1fd37fe640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.938+0000 7f1fd37fe640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.938+0000 7f1fd37fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.938+0000 7f1fd37fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.938+0000 7f1fd37fe640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fbc097840 0x7f1fbc099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.938+0000 7f1fd37fe640 1 -- 172.21.15.133:0/1318971459 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f1fd40c0be0 con 0x7f1fdc0a2cf0 2024-09-16T15:07:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.938+0000 7f1fdad9d640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fbc097840 0x7f1fbc099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:26.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.938+0000 7f1fdad9d640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fbc097840 0x7f1fbc099d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0af000 tx=0x7f1fcc0a7000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:26.942+0000 7f1fd37fe640 1 -- 172.21.15.133:0/1318971459 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1fd408e2f0 con 0x7f1fdc0a2cf0 2024-09-16T15:07:27.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.086+0000 7f1fe1008640 1 -- 172.21.15.133:0/1318971459 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fdc140380 con 0x7f1fdc0a2cf0 2024-09-16T15:07:27.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.086+0000 7f1fd37fe640 1 -- 172.21.15.133:0/1318971459 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+13441 (secure 0 0 0) 0x7f1fd401a090 con 0x7f1fdc0a2cf0 2024-09-16T15:07:27.092 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:27.092 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:23.049976+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:07:20.523939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-16T15:07:21.043356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 10 mgrc shutdown 2024-09-16T15:07:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 -- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fbc097840 msgr2=0x7f1fbc099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fbc097840 0x7f1fbc099d00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0af000 tx=0x7f1fcc0a7000 comp rx=0 tx=0).stop 2024-09-16T15:07:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 -- 172.21.15.133:0/1318971459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 msgr2=0x7f1fdc139aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc139aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400cef0 tx=0x7f1fd400a710 comp rx=0 tx=0).stop 2024-09-16T15:07:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 -- 172.21.15.133:0/1318971459 shutdown_connections 2024-09-16T15:07:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1fdc0a2cf0 0x7f1fdc139aa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1fdc0a1010 0x7f1fdc1415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fbc097840 0x7f1fbc099d00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 --2- 172.21.15.133:0/1318971459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fdc09e490 0x7f1fdc145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 -- 172.21.15.133:0/1318971459 >> 172.21.15.133:0/1318971459 conn(0x7f1fdc0a3fd0 msgr2=0x7f1fdc0b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 -- 172.21.15.133:0/1318971459 shutdown_connections 2024-09-16T15:07:27.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.090+0000 7f1fe1008640 1 -- 172.21.15.133:0/1318971459 wait complete. 2024-09-16T15:07:27.109 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T15:07:27.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T15:07:27.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T15:07:27.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T15:07:27.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.210+0000 7ff38f6e8640 1 Processor -- start 2024-09-16T15:07:27.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.210+0000 7f5773254640 1 Processor -- start 2024-09-16T15:07:27.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.210+0000 7ff38f6e8640 1 -- start start 2024-09-16T15:07:27.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.210+0000 7f5773254640 1 -- start start 2024-09-16T15:07:27.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f5773254640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 0x7f576c143590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f5773254640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f576c143ad0 0x7f576c13f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f5773254640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576c143f60 con 0x7f576c09f490 2024-09-16T15:07:27.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f5773254640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c1444e0 con 0x7f576c143ad0 2024-09-16T15:07:27.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f5773254640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c144220 con 0x7f576c0a7540 2024-09-16T15:07:27.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f576bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f576c143ad0 0x7f576c13f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f5770ff0640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 0x7f576c143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f576bfff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f576c143ad0 0x7f576c13f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57248/0 (socket says 172.21.15.133:57248) 2024-09-16T15:07:27.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f576bfff640 1 -- 172.21.15.133:0/2661587077 learned_addr learned my addr 172.21.15.133:0/2661587077 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38f6e8640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff3880a78a0 0x7ff3880a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38f6e8640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff3881408e0 0x7ff388140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38f6e8640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff388141200 con 0x7ff3881435f0 2024-09-16T15:07:27.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38f6e8640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3880a3ad0 con 0x7ff3881408e0 2024-09-16T15:07:27.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38f6e8640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3880a2de0 con 0x7ff3880a78a0 2024-09-16T15:07:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f576bfff640 1 -- 172.21.15.133:0/2661587077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 msgr2=0x7f576c143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f576bfff640 1 --2- 172.21.15.133:0/2661587077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 0x7f576c143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f576bfff640 1 -- 172.21.15.133:0/2661587077 >> v1:172.21.15.133:6789/0 conn(0x7f576c09f490 legacy=0x7f576c13fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f576bfff640 1 -- 172.21.15.133:0/2661587077 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c0a3630 con 0x7f576c143ad0 2024-09-16T15:07:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38cc83640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff3881408e0 0x7ff388140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38cc83640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff3881408e0 0x7ff388140cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.133:46824/0 (socket says 172.21.15.133:46824) 2024-09-16T15:07:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38cc83640 1 -- 172.21.15.133:0/2476005806 learned_addr learned my addr 172.21.15.133:0/2476005806 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38dc85640 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7ff3881435f0 0x7ff388143aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.133:54566/0 (socket says 172.21.15.133:54566) 2024-09-16T15:07:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38d484640 1 --2- 172.21.15.133:0/2476005806 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff3880a78a0 0x7ff3880a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38d484640 1 -- 172.21.15.133:0/2476005806 >> v2:172.21.15.191:3300/0 conn(0x7ff3881408e0 msgr2=0x7ff388140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38d484640 1 --2- 172.21.15.133:0/2476005806 >> v2:172.21.15.191:3300/0 conn(0x7ff3881408e0 0x7ff388140cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38d484640 1 -- 172.21.15.133:0/2476005806 >> v1:172.21.15.191:6789/0 conn(0x7ff3881435f0 legacy=0x7ff388143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7ff38d484640 1 -- 172.21.15.133:0/2476005806 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3881441b0 con 0x7ff3880a78a0 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08fb52c640 1 Processor -- start 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08fb52c640 1 -- start start 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08fb52c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f08f414ce40 0x7f08f414f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08fb52c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f406c870 con 0x7f08f406b380 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08fb52c640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f406cb30 con 0x7f08f4143de0 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08fb52c640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f40a6d80 con 0x7f08f414ce40 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08f92c8640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f08f406b380 0x7f08f40a6670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33898/0 (socket says 172.21.15.133:33898) 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08f92c8640 1 -- 172.21.15.133:0/3261955259 learned_addr learned my addr 172.21.15.133:0/3261955259 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08f9ac9640 1 --2- 172.21.15.133:0/3261955259 >> v2:172.21.15.191:3300/0 conn(0x7f08f414ce40 0x7f08f414f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984270874 0 0) 0x7f08f406c870 con 0x7f08f406b380 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.214+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08e0003530 con 0x7f08f406b380 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2939542732 0 0) 0x7f08e0003530 con 0x7f08f406b380 2024-09-16T15:07:27.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 >> v2:172.21.15.191:3300/0 conn(0x7f08f414ce40 msgr2=0x7f08f414f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 --2- 172.21.15.133:0/3261955259 >> v2:172.21.15.191:3300/0 conn(0x7f08f414ce40 0x7f08f414f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 >> v1:172.21.15.191:6789/0 conn(0x7f08f4143de0 legacy=0x7f08f41441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40a6010 con 0x7f08f406b380 2024-09-16T15:07:27.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f08f00016a0 con 0x7f08f406b380 2024-09-16T15:07:27.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f576bfff640 1 --2- 172.21.15.133:0/2661587077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576c143ad0 0x7f576c13f770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5758009870 tx=0x7f5758020860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c48097209c2562 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 >> v1:172.21.15.133:6789/0 conn(0x7f08f406b380 legacy=0x7f08f40a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08e0003980 0x7f08e0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08e00047d0 0x7f08e0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08e00052b0 0x7f08e000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40a6d80 con 0x7f08e00052b0 2024-09-16T15:07:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e000c180 con 0x7f08e00047d0 2024-09-16T15:07:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e000c340 con 0x7f08e0003980 2024-09-16T15:07:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08f92c8640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08e0003980 0x7f08e0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08f8ac7640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08e00047d0 0x7f08e0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08f9ac9640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08e00052b0 0x7f08e000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f576b7fe640 1 -- 172.21.15.133:0/2661587077 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580a9820 con 0x7f576c143ad0 2024-09-16T15:07:27.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f576b7fe640 1 -- 172.21.15.133:0/2661587077 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57580a9980 con 0x7f576c143ad0 2024-09-16T15:07:27.226 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f576b7fe640 1 -- 172.21.15.133:0/2661587077 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580a9d10 con 0x7f576c143ad0 2024-09-16T15:07:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38d484640 1 --2- 172.21.15.133:0/2476005806 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff3880a78a0 0x7ff3880a7c80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff3840097c0 tx=0x7ff38401c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe051be10ac24a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff37ffff640 1 -- 172.21.15.133:0/2476005806 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3840ab600 con 0x7ff3880a78a0 2024-09-16T15:07:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff37ffff640 1 -- 172.21.15.133:0/2476005806 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3840a7050 con 0x7ff3880a78a0 2024-09-16T15:07:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff37ffff640 1 -- 172.21.15.133:0/2476005806 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3840ab980 con 0x7ff3880a78a0 2024-09-16T15:07:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f5773254640 1 -- 172.21.15.133:0/2661587077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576c143ad0 msgr2=0x7f576c13f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f5773254640 1 --2- 172.21.15.133:0/2661587077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576c143ad0 0x7f576c13f770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5758009870 tx=0x7f5758020860 comp rx=0 tx=0).stop 2024-09-16T15:07:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08f9ac9640 1 -- 172.21.15.133:0/3261955259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08e0003980 msgr2=0x7f08e0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08f9ac9640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08e0003980 0x7f08e0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08f9ac9640 1 -- 172.21.15.133:0/3261955259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08e00047d0 msgr2=0x7f08e0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08f9ac9640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08e00047d0 0x7f08e0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f5773254640 1 -- 172.21.15.133:0/2661587077 shutdown_connections 2024-09-16T15:07:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f5773254640 1 --2- 172.21.15.133:0/2661587077 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576c143ad0 0x7f576c13f770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f5773254640 1 --2- 172.21.15.133:0/2661587077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 0x7f576c143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f5773254640 1 -- 172.21.15.133:0/2661587077 >> 172.21.15.133:0/2661587077 conn(0x7f576c0a4f70 msgr2=0x7f576c0a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08f9ac9640 1 -- 172.21.15.133:0/3261955259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e0003530 con 0x7f08e00052b0 2024-09-16T15:07:27.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2476005806 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff3880a78a0 msgr2=0x7ff3880a7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 --2- 172.21.15.133:0/2476005806 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff3880a78a0 0x7ff3880a7c80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff3840097c0 tx=0x7ff38401c360 comp rx=0 tx=0).stop 2024-09-16T15:07:27.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2476005806 shutdown_connections 2024-09-16T15:07:27.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 --2- 172.21.15.133:0/2476005806 >> v2:172.21.15.191:3300/0 conn(0x7ff3881408e0 0x7ff388140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 --2- 172.21.15.133:0/2476005806 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff3880a78a0 0x7ff3880a7c80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2476005806 >> 172.21.15.133:0/2476005806 conn(0x7ff3880a51b0 msgr2=0x7ff3880a55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2476005806 shutdown_connections 2024-09-16T15:07:27.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2476005806 wait complete. 2024-09-16T15:07:27.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 Processor -- start 2024-09-16T15:07:27.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08f9ac9640 1 -- 172.21.15.133:0/3261955259 reap_dead start 2024-09-16T15:07:27.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f5773254640 1 -- 172.21.15.133:0/2661587077 shutdown_connections 2024-09-16T15:07:27.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08f9ac9640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08e00052b0 0x7f08e000b990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009f90 tx=0x7f08ec01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f72ad32a5e09cd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec003ba0 con 0x7f08e00052b0 2024-09-16T15:07:27.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec00f040 con 0x7f08e00052b0 2024-09-16T15:07:27.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7f08ebfff640 1 -- 172.21.15.133:0/3261955259 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0a5860 con 0x7f08e00052b0 2024-09-16T15:07:27.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 -- start start 2024-09-16T15:07:27.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.218+0000 7ff38f6e8640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff3880a78a0 0x7ff38814b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38f6e8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3881408e0 0x7ff3880b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38f6e8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3881435f0 0x7ff3880707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38f6e8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38814b8f0 con 0x7ff3881408e0 2024-09-16T15:07:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38f6e8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38814bbb0 con 0x7ff3881435f0 2024-09-16T15:07:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38f6e8640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff388071bd0 con 0x7ff3880a78a0 2024-09-16T15:07:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5773254640 1 -- 172.21.15.133:0/2661587077 wait complete. 2024-09-16T15:07:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38dc85640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3881435f0 0x7ff3880707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38dc85640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3881435f0 0x7ff3880707f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.133:46854/0 (socket says 172.21.15.133:46854) 2024-09-16T15:07:27.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38dc85640 1 -- 172.21.15.133:0/2621923828 learned_addr learned my addr 172.21.15.133:0/2621923828 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 -- 172.21.15.133:0/3261955259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08e00052b0 msgr2=0x7f08e000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08e00052b0 0x7f08e000b990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009f90 tx=0x7f08ec01cc60 comp rx=0 tx=0).stop 2024-09-16T15:07:27.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 -- 172.21.15.133:0/3261955259 shutdown_connections 2024-09-16T15:07:27.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 --2- 172.21.15.133:0/3261955259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08e00052b0 0x7f08e000b990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 -- 172.21.15.133:0/3261955259 >> 172.21.15.133:0/3261955259 conn(0x7f08f406f000 msgr2=0x7f08f406f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38dc85640 1 -- 172.21.15.133:0/2621923828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff3880a78a0 msgr2=0x7ff38814b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38dc85640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff3880a78a0 0x7ff38814b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38dc85640 1 -- 172.21.15.133:0/2621923828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3881408e0 msgr2=0x7ff3880b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38dc85640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3881408e0 0x7ff3880b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38dc85640 1 -- 172.21.15.133:0/2621923828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff384009420 con 0x7ff3881435f0 2024-09-16T15:07:27.234 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 -- 172.21.15.133:0/3261955259 shutdown_connections 2024-09-16T15:07:27.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 -- 172.21.15.133:0/3261955259 wait complete. 2024-09-16T15:07:27.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38dc85640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3881435f0 0x7ff3880707f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff38000c700 tx=0x7ff38000cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5773254640 1 Processor -- start 2024-09-16T15:07:27.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5773254640 1 -- start start 2024-09-16T15:07:27.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5773254640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f576c09f490 0x7f576c0a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5773254640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 0x7f576c0b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 Processor -- start 2024-09-16T15:07:27.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5773254640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576c1dad80 0x7f576c1db210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5773254640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0b5b10 con 0x7f576c1dad80 2024-09-16T15:07:27.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5773254640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0722b0 con 0x7f576c09f490 2024-09-16T15:07:27.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5773254640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c1d7710 con 0x7f576c0a7540 2024-09-16T15:07:27.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 -- start start 2024-09-16T15:07:27.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff37dffb640 1 -- 172.21.15.133:0/2621923828 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38000ce10 con 0x7ff3881435f0 2024-09-16T15:07:27.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff37dffb640 1 -- 172.21.15.133:0/2621923828 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38000a640 con 0x7ff3881435f0 2024-09-16T15:07:27.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff37dffb640 1 -- 172.21.15.133:0/2621923828 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38000a9b0 con 0x7ff3881435f0 2024-09-16T15:07:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f576bfff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 0x7f576c0b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f576bfff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 0x7f576c0b6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:52894/0 (socket says 172.21.15.133:52894) 2024-09-16T15:07:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f576bfff640 1 -- 172.21.15.133:0/2759046001 learned_addr learned my addr 172.21.15.133:0/2759046001 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f57717f1640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576c1dad80 0x7f576c1db210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5770ff0640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f576c09f490 0x7f576c0a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f576bfff640 1 -- 172.21.15.133:0/2759046001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f576c09f490 msgr2=0x7f576c0a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f576bfff640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f576c09f490 0x7f576c0a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f576bfff640 1 -- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576c1dad80 msgr2=0x7f576c1db210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f576bfff640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f576c1dad80 0x7f576c1db210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f576bfff640 1 -- 172.21.15.133:0/2759046001 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57580094d0 con 0x7f576c0a7540 2024-09-16T15:07:27.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f5770ff0640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f576c09f490 0x7f576c0a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:27.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08e00052b0 0x7f08f40b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08f414e270 0x7f08f4138850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f4138d90 0x7f08f4139210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40b5d90 con 0x7f08f414e270 2024-09-16T15:07:27.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40b6650 con 0x7f08f4138d90 2024-09-16T15:07:27.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08fb52c640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40b6910 con 0x7f08e00052b0 2024-09-16T15:07:27.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f576bfff640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 0x7f576c0b6720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57580b4040 tx=0x7f575801d600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08f8ac7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08f414e270 0x7f08f4138850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08f8ac7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08f414e270 0x7f08f4138850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57280/0 (socket says 172.21.15.133:57280) 2024-09-16T15:07:27.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08f8ac7640 1 -- 172.21.15.133:0/275733104 learned_addr learned my addr 172.21.15.133:0/275733104 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08f8ac7640 1 -- 172.21.15.133:0/275733104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08e00052b0 msgr2=0x7f08f40b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08f8ac7640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08e00052b0 0x7f08f40b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08f8ac7640 1 -- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f4138d90 msgr2=0x7f08f4139210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:07:27.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08f8ac7640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f4138d90 0x7f08f4139210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7f08f8ac7640 1 -- 172.21.15.133:0/275733104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f4147990 con 0x7f08f414e270 2024-09-16T15:07:27.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38814b600 con 0x7ff3881435f0 2024-09-16T15:07:27.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.222+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38814ca60 con 0x7ff3881435f0 2024-09-16T15:07:27.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f08f8ac7640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08f414e270 0x7f08f4138850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08e400f7f0 tx=0x7f08e400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f08e9ffb640 1 -- 172.21.15.133:0/275733104 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e40050d0 con 0x7f08f414e270 2024-09-16T15:07:27.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f08e9ffb640 1 -- 172.21.15.133:0/275733104 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e4005230 con 0x7f08f414e270 2024-09-16T15:07:27.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f08e9ffb640 1 -- 172.21.15.133:0/275733104 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400d730 con 0x7f08f414e270 2024-09-16T15:07:27.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3880a7c80 con 0x7ff3881435f0 2024-09-16T15:07:27.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f57697fa640 1 -- 172.21.15.133:0/2759046001 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580b1850 con 0x7f576c0a7540 2024-09-16T15:07:27.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f57697fa640 1 -- 172.21.15.133:0/2759046001 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57580b19b0 con 0x7f576c0a7540 2024-09-16T15:07:27.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c1dba10 con 0x7f576c0a7540 2024-09-16T15:07:27.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f57697fa640 1 -- 172.21.15.133:0/2759046001 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580b1d60 con 0x7f576c0a7540 2024-09-16T15:07:27.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c071f20 con 0x7f576c0a7540 2024-09-16T15:07:27.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f40b7e30 con 0x7f08f414e270 2024-09-16T15:07:27.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f40b6250 con 0x7f08f414e270 2024-09-16T15:07:27.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.226+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08f40a6670 con 0x7f08f414e270 2024-09-16T15:07:27.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 1 -- 172.21.15.133:0/2759046001 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f57580bc480 con 0x7f576c0a7540 2024-09-16T15:07:27.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:27.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08e9ffb640 1 -- 172.21.15.133:0/275733104 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f08e400d890 con 0x7f08f414e270 2024-09-16T15:07:27.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08e9ffb640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:27.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f576c072110 con 0x7f576c0a7540 2024-09-16T15:07:27.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:27.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5750097910 0x7f5750099dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08e9ffb640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:27.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08e9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08e9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08e9ffb640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f08d8097840 0x7f08d8099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 1 -- 172.21.15.133:0/2759046001 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f575815eb30 con 0x7f576c0a7540 2024-09-16T15:07:27.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08e9ffb640 1 -- 172.21.15.133:0/275733104 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f08e40c0670 con 0x7f08f414e270 2024-09-16T15:07:27.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:6800/191622751,v1:172.21.15.133:6801/191622751] conn(0x7f575009f4e0 0x7f57500a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 1 -- 172.21.15.133:0/2759046001 --> [v2:172.21.15.133:6800/191622751,v1:172.21.15.133:6801/191622751] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57500a2010 con 0x7f575009f4e0 2024-09-16T15:07:27.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 1 -- 172.21.15.133:0/2759046001 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f575812c240 con 0x7f576c0a7540 2024-09-16T15:07:27.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08e9ffb640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:6816/1482439914,v1:172.21.15.191:6817/1482439914] conn(0x7f08d809f410 0x7f08d80a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08e9ffb640 1 -- 172.21.15.133:0/275733104 --> [v2:172.21.15.191:6816/1482439914,v1:172.21.15.191:6817/1482439914] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08d80a1f40 con 0x7f08d809f410 2024-09-16T15:07:27.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08e9ffb640 1 -- 172.21.15.133:0/275733104 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08e40c0ac0 con 0x7f08f414e270 2024-09-16T15:07:27.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57717f1640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:6800/191622751,v1:172.21.15.133:6801/191622751] conn(0x7f575009f4e0 0x7f57500a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08f92c8640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f08d8097840 0x7f08d8099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57717f1640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:6800/191622751,v1:172.21.15.133:6801/191622751] conn(0x7f575009f4e0 0x7f57500a1940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f5770ff0640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5750097910 0x7f5750099dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08f9ac9640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:6816/1482439914,v1:172.21.15.191:6817/1482439914] conn(0x7f08d809f410 0x7f08d80a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f57697fa640 1 -- 172.21.15.133:0/2759046001 <== osd.0 v2:172.21.15.133:6800/191622751 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f57500a2010 con 0x7f575009f4e0 2024-09-16T15:07:27.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08f92c8640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f08d8097840 0x7f08d8099d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08f0001210 tx=0x7f08f0001160 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f08f9ac9640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:6816/1482439914,v1:172.21.15.191:6817/1482439914] conn(0x7f08d809f410 0x7f08d80a1870 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f9cf6327640 1 Processor -- start 2024-09-16T15:07:27.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.230+0000 7f9cf6327640 1 -- start start 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cf6327640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9cf0065d00 0x7f9cf00a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cf6327640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9cf00660e0 0x7f9cf006b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cf6327640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf00a6f20 con 0x7f9cf00a74a0 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cf6327640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf006b180 con 0x7f9cf00660e0 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cf6327640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf006b440 con 0x7f9cf0065d00 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cf48c4640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9cf00a74a0 0x7f9cf01469a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33912/0 (socket says 172.21.15.133:33912) 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cf48c4640 1 -- 172.21.15.133:0/663030163 learned_addr learned my addr 172.21.15.133:0/663030163 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cef7fe640 1 --2- 172.21.15.133:0/663030163 >> v2:172.21.15.133:3300/0 conn(0x7f9cf00660e0 0x7f9cf006b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cef7fe640 1 -- 172.21.15.133:0/663030163 >> v2:172.21.15.191:3300/0 conn(0x7f9cf0065d00 msgr2=0x7f9cf00a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cef7fe640 1 --2- 172.21.15.133:0/663030163 >> v2:172.21.15.191:3300/0 conn(0x7f9cf0065d00 0x7f9cf00a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cef7fe640 1 -- 172.21.15.133:0/663030163 >> v1:172.21.15.133:6789/0 conn(0x7f9cf00a74a0 legacy=0x7f9cf01469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9ceeffd640 1 -- 172.21.15.133:0/663030163 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530152219 0 0) 0x7f9cf00a6f20 con 0x7f9cf00a74a0 2024-09-16T15:07:27.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cef7fe640 1 -- 172.21.15.133:0/663030163 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf00a6b10 con 0x7f9cf00660e0 2024-09-16T15:07:27.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f5770ff0640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5750097910 0x7f5750099dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5760009ae0 tx=0x7f5760008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f08e9ffb640 1 -- 172.21.15.133:0/275733104 <== osd.1 v2:172.21.15.191:6816/1482439914 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f08d80a1f40 con 0x7f08d809f410 2024-09-16T15:07:27.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff37dffb640 1 -- 172.21.15.133:0/2621923828 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7ff38000ab10 con 0x7ff3881435f0 2024-09-16T15:07:27.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff37dffb640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:27.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff37dffb640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:27.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff37dffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff37dffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff37dffb640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff36c097870 0x7ff36c099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff37dffb640 1 -- 172.21.15.133:0/2621923828 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7ff3800bfc40 con 0x7ff3881435f0 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff37dffb640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] conn(0x7ff36c09f440 0x7ff36c0a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff37dffb640 1 -- 172.21.15.133:0/2621923828 --> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff36c0a1f70 con 0x7ff36c09f440 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff37dffb640 1 -- 172.21.15.133:0/2621923828 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3800bffb0 con 0x7ff3881435f0 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff38d484640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff36c097870 0x7ff36c099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9cef7fe640 1 --2- 172.21.15.133:0/663030163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf00660e0 0x7f9cf006b890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0066530 tx=0x7f9ce401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5531ef7f2e2a2df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9ceeffd640 1 -- 172.21.15.133:0/663030163 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40ab600 con 0x7f9cf00660e0 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9ceeffd640 1 -- 172.21.15.133:0/663030163 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce40a7050 con 0x7f9cf00660e0 2024-09-16T15:07:27.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7f9ceeffd640 1 -- 172.21.15.133:0/663030163 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40ab980 con 0x7f9cf00660e0 2024-09-16T15:07:27.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.234+0000 7ff38cc83640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] conn(0x7ff36c09f440 0x7ff36c0a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- 172.21.15.133:0/663030163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf00660e0 msgr2=0x7f9cf006b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 --2- 172.21.15.133:0/663030163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf00660e0 0x7f9cf006b890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0066530 tx=0x7f9ce401c3a0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7ff38d484640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff36c097870 0x7ff36c099d30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff3840ad000 tx=0x7ff384004740 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7ff38cc83640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] conn(0x7ff36c09f440 0x7ff36c0a18a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- 172.21.15.133:0/663030163 shutdown_connections 2024-09-16T15:07:27.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 --2- 172.21.15.133:0/663030163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf00660e0 0x7f9cf006b890 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 --2- 172.21.15.133:0/663030163 >> v2:172.21.15.191:3300/0 conn(0x7f9cf0065d00 0x7f9cf00a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- 172.21.15.133:0/663030163 >> 172.21.15.133:0/663030163 conn(0x7f9cf006a030 msgr2=0x7f9cf006a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- 172.21.15.133:0/663030163 shutdown_connections 2024-09-16T15:07:27.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- 172.21.15.133:0/663030163 wait complete. 2024-09-16T15:07:27.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 Processor -- start 2024-09-16T15:07:27.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- start start 2024-09-16T15:07:27.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0065d00 0x7f9cf0070e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9cf00a74a0 0x7f9cf0144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7ff37dffb640 1 -- 172.21.15.133:0/2621923828 <== osd.3 v2:172.21.15.191:6800/787707562 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff36c0a1f70 con 0x7ff36c09f440 2024-09-16T15:07:27.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006a710 0x7f9cf01df420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf01451e0 con 0x7f9cf0065d00 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00a1cd0 con 0x7f9cf00a74a0 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf01453f0 con 0x7f9cf006a710 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9ceffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0065d00 0x7f9cf0070e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9ceffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0065d00 0x7f9cf0070e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57300/0 (socket says 172.21.15.133:57300) 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9ceffff640 1 -- 172.21.15.133:0/393064847 learned_addr learned my addr 172.21.15.133:0/393064847 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf48c4640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006a710 0x7f9cf01df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cef7fe640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9cf00a74a0 0x7f9cf0144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9ceffff640 1 -- 172.21.15.133:0/393064847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006a710 msgr2=0x7f9cf01df420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9ceffff640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006a710 0x7f9cf01df420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9ceffff640 1 -- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9cf00a74a0 msgr2=0x7f9cf0144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9ceffff640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9cf00a74a0 0x7f9cf0144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9ceffff640 1 -- 172.21.15.133:0/393064847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4009420 con 0x7f9cf0065d00 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9ceffff640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0065d00 0x7f9cf0070e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000b440 tx=0x7f9ce000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cecff9640 1 -- 172.21.15.133:0/393064847 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0011890 con 0x7f9cf0065d00 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cecff9640 1 -- 172.21.15.133:0/393064847 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0011a10 con 0x7f9cf0065d00 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- 172.21.15.133:0/393064847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf0072540 con 0x7f9cf0065d00 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cecff9640 1 -- 172.21.15.133:0/393064847 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0011dc0 con 0x7f9cf0065d00 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.238+0000 7f9cf6327640 1 -- 172.21.15.133:0/393064847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf0142b10 con 0x7f9cf0065d00 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cd67fc640 1 -- 172.21.15.133:0/393064847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9cc8000f20 con 0x7f9cf0065d00 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cecff9640 1 -- 172.21.15.133:0/393064847 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f9ce001a400 con 0x7f9cf0065d00 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cecff9640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cecff9640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cecff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cecff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cecff9640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9cdc097870 0x7f9cdc099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cecff9640 1 -- 172.21.15.133:0/393064847 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f9ce00c0080 con 0x7f9cf0065d00 2024-09-16T15:07:27.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cecff9640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:6801/193816588,v1:172.21.15.191:6802/193816588] conn(0x7f9cdc09f2d0 0x7f9cdc0a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cecff9640 1 -- 172.21.15.133:0/393064847 --> [v2:172.21.15.191:6801/193816588,v1:172.21.15.191:6802/193816588] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9cdc0a1e00 con 0x7f9cdc09f2d0 2024-09-16T15:07:27.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cecff9640 1 -- 172.21.15.133:0/393064847 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce00c2740 con 0x7f9cf0065d00 2024-09-16T15:07:27.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cef7fe640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9cdc097870 0x7f9cdc099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cf48c4640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:6801/193816588,v1:172.21.15.191:6802/193816588] conn(0x7f9cdc09f2d0 0x7f9cdc0a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cf48c4640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:6801/193816588,v1:172.21.15.191:6802/193816588] conn(0x7f9cdc09f2d0 0x7f9cdc0a1730 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.242+0000 7f9cef7fe640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9cdc097870 0x7f9cdc099d30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40ad000 tx=0x7f9ce4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.246+0000 7f9cecff9640 1 -- 172.21.15.133:0/393064847 <== osd.2 v2:172.21.15.191:6801/193816588 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9cdc0a1e00 con 0x7f9cdc09f2d0 2024-09-16T15:07:27.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.254+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 --> [v2:172.21.15.133:6800/191622751,v1:172.21.15.133:6801/191622751] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f576c140e20 con 0x7f575009f4e0 2024-09-16T15:07:27.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f57697fa640 1 -- 172.21.15.133:0/2759046001 <== osd.0 v2:172.21.15.133:6800/191622751 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5764001aa0 con 0x7f575009f4e0 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:6800/191622751,v1:172.21.15.133:6801/191622751] conn(0x7f575009f4e0 msgr2=0x7f57500a1940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:6800/191622751,v1:172.21.15.133:6801/191622751] conn(0x7f575009f4e0 0x7f57500a1940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 10 mgrc shutdown 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5750097910 msgr2=0x7f5750099dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5750097910 0x7f5750099dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5760009ae0 tx=0x7f5760008040 comp rx=0 tx=0).stop 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 msgr2=0x7f576c0b6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 1 --2- 172.21.15.133:0/2759046001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f576c0a7540 0x7f576c0b6720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f57580b4040 tx=0x7f575801d600 comp rx=0 tx=0).stop 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f57717f1640 1 -- 172.21.15.133:0/2759046001 reap_dead start 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 shutdown_connections 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 >> 172.21.15.133:0/2759046001 conn(0x7f576c0a4f70 msgr2=0x7f576c13f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 shutdown_connections 2024-09-16T15:07:27.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f5773254640 1 -- 172.21.15.133:0/2759046001 wait complete. 2024-09-16T15:07:27.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 --> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff38813b1a0 con 0x7ff36c09f440 2024-09-16T15:07:27.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f9cd67fc640 1 -- 172.21.15.133:0/393064847 --> [v2:172.21.15.191:6801/193816588,v1:172.21.15.191:6802/193816588] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9cc8002b60 con 0x7f9cdc09f2d0 2024-09-16T15:07:27.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7ff37dffb640 1 -- 172.21.15.133:0/2621923828 <== osd.3 v2:172.21.15.191:6800/787707562 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff38813b1a0 con 0x7ff36c09f440 2024-09-16T15:07:27.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] conn(0x7ff36c09f440 msgr2=0x7ff36c0a18a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7ff38f6e8640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:6800/787707562,v1:172.21.15.191:6804/787707562] conn(0x7ff36c09f440 0x7ff36c0a18a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f9cecff9640 1 -- 172.21.15.133:0/393064847 <== osd.2 v2:172.21.15.191:6801/193816588 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9cc8002b60 con 0x7f9cdc09f2d0 2024-09-16T15:07:27.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7ff38f6e8640 10 mgrc shutdown 2024-09-16T15:07:27.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f9cd67fc640 1 -- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:6801/193816588,v1:172.21.15.191:6802/193816588] conn(0x7f9cdc09f2d0 msgr2=0x7f9cdc0a1730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.258+0000 7f9cd67fc640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:6801/193816588,v1:172.21.15.191:6802/193816588] conn(0x7f9cdc09f2d0 0x7f9cdc0a1730 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f9cd67fc640 10 mgrc shutdown 2024-09-16T15:07:27.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff36c097870 msgr2=0x7ff36c099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7ff38f6e8640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff36c097870 0x7ff36c099d30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff3840ad000 tx=0x7ff384004740 comp rx=0 tx=0).stop 2024-09-16T15:07:27.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3881435f0 msgr2=0x7ff3880707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7ff38f6e8640 1 --2- 172.21.15.133:0/2621923828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3881435f0 0x7ff3880707f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff38000c700 tx=0x7ff38000cbc0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7ff38dc85640 1 -- 172.21.15.133:0/2621923828 reap_dead start 2024-09-16T15:07:27.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f9cd67fc640 1 -- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9cdc097870 msgr2=0x7f9cdc099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f9cd67fc640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9cdc097870 0x7f9cdc099d30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40ad000 tx=0x7f9ce4008040 comp rx=0 tx=0).stop 2024-09-16T15:07:27.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f9cd67fc640 1 -- 172.21.15.133:0/393064847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0065d00 msgr2=0x7f9cf0070e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 --> [v2:172.21.15.191:6816/1482439914,v1:172.21.15.191:6817/1482439914] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08f40a6670 con 0x7f08d809f410 2024-09-16T15:07:27.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f9cd67fc640 1 --2- 172.21.15.133:0/393064847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0065d00 0x7f9cf0070e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000b440 tx=0x7f9ce000b900 comp rx=0 tx=0).stop 2024-09-16T15:07:27.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f9cf48c4640 1 -- 172.21.15.133:0/393064847 reap_dead start 2024-09-16T15:07:27.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f9cd67fc640 1 -- 172.21.15.133:0/393064847 shutdown_connections 2024-09-16T15:07:27.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f9cd67fc640 1 -- 172.21.15.133:0/393064847 >> 172.21.15.133:0/393064847 conn(0x7f9cf006a030 msgr2=0x7f9cf00a2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 shutdown_connections 2024-09-16T15:07:27.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 >> 172.21.15.133:0/2621923828 conn(0x7ff3880a51b0 msgr2=0x7ff388144c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f9cd67fc640 1 -- 172.21.15.133:0/393064847 shutdown_connections 2024-09-16T15:07:27.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f9cd67fc640 1 -- 172.21.15.133:0/393064847 wait complete. 2024-09-16T15:07:27.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 shutdown_connections 2024-09-16T15:07:27.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7ff38f6e8640 1 -- 172.21.15.133:0/2621923828 wait complete. 2024-09-16T15:07:27.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08e9ffb640 1 -- 172.21.15.133:0/275733104 <== osd.1 v2:172.21.15.191:6816/1482439914 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08ec0a6970 con 0x7f08d809f410 2024-09-16T15:07:27.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:6816/1482439914,v1:172.21.15.191:6817/1482439914] conn(0x7f08d809f410 msgr2=0x7f08d80a1870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:6816/1482439914,v1:172.21.15.191:6817/1482439914] conn(0x7f08d809f410 0x7f08d80a1870 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 10 mgrc shutdown 2024-09-16T15:07:27.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f08d8097840 msgr2=0x7f08d8099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f08d8097840 0x7f08d8099d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08f0001210 tx=0x7f08f0001160 comp rx=0 tx=0).stop 2024-09-16T15:07:27.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08f414e270 msgr2=0x7f08f4138850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 --2- 172.21.15.133:0/275733104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08f414e270 0x7f08f4138850 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08e400f7f0 tx=0x7f08e400fcb0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08f9ac9640 1 -- 172.21.15.133:0/275733104 reap_dead start 2024-09-16T15:07:27.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 shutdown_connections 2024-09-16T15:07:27.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 >> 172.21.15.133:0/275733104 conn(0x7f08f406f000 msgr2=0x7f08f406ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 shutdown_connections 2024-09-16T15:07:27.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.262+0000 7f08fb52c640 1 -- 172.21.15.133:0/275733104 wait complete. 2024-09-16T15:07:27.283 INFO:teuthology.orchestra.run.smithi133.stdout:38654705670 2024-09-16T15:07:27.284 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T15:07:27.287 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-09-16T15:07:27.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T15:07:27.290 INFO:teuthology.orchestra.run.smithi133.stdout:38654705670 2024-09-16T15:07:27.291 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T15:07:27.292 INFO:teuthology.orchestra.run.smithi133.stdout:38654705670 2024-09-16T15:07:27.292 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T15:07:27.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.382+0000 7f200740b640 1 Processor -- start 2024-09-16T15:07:27.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.382+0000 7f200740b640 1 -- start start 2024-09-16T15:07:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.382+0000 7f200740b640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f200006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.382+0000 7f200740b640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2000147070 con 0x7f20001469b0 2024-09-16T15:07:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.382+0000 7f200740b640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2000147330 con 0x7f20000a4170 2024-09-16T15:07:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.382+0000 7f200740b640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2000146e60 con 0x7f200006b260 2024-09-16T15:07:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.386+0000 7f20059a8640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f20001469b0 0x7f200014dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33924/0 (socket says 172.21.15.133:33924) 2024-09-16T15:07:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.386+0000 7f20059a8640 1 -- 172.21.15.133:0/2001693133 learned_addr learned my addr 172.21.15.133:0/2001693133 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.386+0000 7f20051a7640 1 --2- 172.21.15.133:0/2001693133 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f200006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.386+0000 7f1ff7fff640 1 -- 172.21.15.133:0/2001693133 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175821514 0 0) 0x7f2000147070 con 0x7f20001469b0 2024-09-16T15:07:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.386+0000 7f1ff7fff640 1 -- 172.21.15.133:0/2001693133 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fec003530 con 0x7f20001469b0 2024-09-16T15:07:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.386+0000 7f20051a7640 1 -- 172.21.15.133:0/2001693133 >> v1:172.21.15.191:6789/0 conn(0x7f20000a4170 legacy=0x7f20001408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.386+0000 7f20051a7640 1 -- 172.21.15.133:0/2001693133 >> v1:172.21.15.133:6789/0 conn(0x7f20001469b0 legacy=0x7f200014dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.386+0000 7f20051a7640 1 -- 172.21.15.133:0/2001693133 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000140fe0 con 0x7f200006b260 2024-09-16T15:07:27.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.386+0000 7f20051a7640 1 --2- 172.21.15.133:0/2001693133 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f200006ca50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0097f0 tx=0x7f1ffc01c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77111160a6d56b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f1ff7fff640 1 -- 172.21.15.133:0/2001693133 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0ab650 con 0x7f200006b260 2024-09-16T15:07:27.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f1ff7fff640 1 -- 172.21.15.133:0/2001693133 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc0ab7b0 con 0x7f200006b260 2024-09-16T15:07:27.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f1ff7fff640 1 -- 172.21.15.133:0/2001693133 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0abb40 con 0x7f200006b260 2024-09-16T15:07:27.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 -- 172.21.15.133:0/2001693133 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 msgr2=0x7f200006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 --2- 172.21.15.133:0/2001693133 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f200006ca50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0097f0 tx=0x7f1ffc01c430 comp rx=0 tx=0).stop 2024-09-16T15:07:27.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 -- 172.21.15.133:0/2001693133 shutdown_connections 2024-09-16T15:07:27.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 --2- 172.21.15.133:0/2001693133 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f200006ca50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 -- 172.21.15.133:0/2001693133 >> 172.21.15.133:0/2001693133 conn(0x7f200006f000 msgr2=0x7f200006f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 -- 172.21.15.133:0/2001693133 shutdown_connections 2024-09-16T15:07:27.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 -- 172.21.15.133:0/2001693133 wait complete. 2024-09-16T15:07:27.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 Processor -- start 2024-09-16T15:07:27.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 -- start start 2024-09-16T15:07:27.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f20000b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20000a4170 0x7f20000af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20001469b0 0x7f200013dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20000b7e20 con 0x7f20000a4170 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200013f180 con 0x7f20001469b0 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.390+0000 7f200740b640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200013f440 con 0x7f200006b260 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20051a7640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f20000b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20051a7640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f20000b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:52928/0 (socket says 172.21.15.133:52928) 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20049a6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20000a4170 0x7f20000af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20051a7640 1 -- 172.21.15.133:0/2399976167 learned_addr learned my addr 172.21.15.133:0/2399976167 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20059a8640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20001469b0 0x7f200013dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20049a6640 1 -- 172.21.15.133:0/2399976167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 msgr2=0x7f20000b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20049a6640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f20000b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20049a6640 1 -- 172.21.15.133:0/2399976167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20001469b0 msgr2=0x7f200013dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20049a6640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20001469b0 0x7f200013dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20049a6640 1 -- 172.21.15.133:0/2399976167 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc009420 con 0x7f20000a4170 2024-09-16T15:07:27.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20051a7640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f20000b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:27.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe416b49640 1 Processor -- start 2024-09-16T15:07:27.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f20049a6640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20000a4170 0x7f20000af8d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0000f70 tx=0x7f1ff000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe416b49640 1 -- start start 2024-09-16T15:07:27.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f1ff5ffb640 1 -- 172.21.15.133:0/2399976167 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00052d0 con 0x7f20000a4170 2024-09-16T15:07:27.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f1ff5ffb640 1 -- 172.21.15.133:0/2399976167 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff0005430 con 0x7f20000a4170 2024-09-16T15:07:27.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f200740b640 1 -- 172.21.15.133:0/2399976167 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20000af0d0 con 0x7f20000a4170 2024-09-16T15:07:27.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f1ff5ffb640 1 -- 172.21.15.133:0/2399976167 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0003840 con 0x7f20000a4170 2024-09-16T15:07:27.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f200740b640 1 -- 172.21.15.133:0/2399976167 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20001387e0 con 0x7f20000a4170 2024-09-16T15:07:27.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe416b49640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe4100a9ae0 0x7fe4100a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe4148e5640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe4101365b0 0x7fe4100aa960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33940/0 (socket says 172.21.15.133:33940) 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe416b49640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 0x7fe4100a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe4148e5640 1 -- 172.21.15.133:0/2599446329 learned_addr learned my addr 172.21.15.133:0/2599446329 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe4150e6640 1 --2- 172.21.15.133:0/2599446329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 0x7fe4100a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe416b49640 1 -- 172.21.15.133:0/2599446329 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe410139b60 con 0x7fe4101365b0 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe416b49640 1 -- 172.21.15.133:0/2599446329 --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4101399f0 con 0x7fe4100a9ae0 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe416b49640 1 -- 172.21.15.133:0/2599446329 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100a5ca0 con 0x7fe4100ab810 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe40ffff640 1 --2- 172.21.15.133:0/2599446329 >> v2:172.21.15.191:3300/0 conn(0x7fe4100a9ae0 0x7fe4100a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe4150e6640 1 -- 172.21.15.133:0/2599446329 >> v2:172.21.15.191:3300/0 conn(0x7fe4100a9ae0 msgr2=0x7fe4100a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe4150e6640 1 --2- 172.21.15.133:0/2599446329 >> v2:172.21.15.191:3300/0 conn(0x7fe4100a9ae0 0x7fe4100a80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe4150e6640 1 -- 172.21.15.133:0/2599446329 >> v1:172.21.15.133:6789/0 conn(0x7fe4101365b0 legacy=0x7fe4100aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f200740b640 1 -- 172.21.15.133:0/2399976167 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20000672a0 con 0x7f20000a4170 2024-09-16T15:07:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f1ff5ffb640 1 -- 172.21.15.133:0/2399976167 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f1ff00039a0 con 0x7f20000a4170 2024-09-16T15:07:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7f1ff5ffb640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe40f7fe640 1 -- 172.21.15.133:0/2599446329 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69781332 0 0) 0x7fe410139b60 con 0x7fe4101365b0 2024-09-16T15:07:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.394+0000 7fe4150e6640 1 -- 172.21.15.133:0/2599446329 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410136990 con 0x7fe4100ab810 2024-09-16T15:07:27.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7f1ff5ffb640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:27.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7f1ff5ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7f1ff5ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7f1ff5ffb640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fe4097870 0x7f1fe4099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7f1ff5ffb640 1 -- 172.21.15.133:0/2399976167 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f1ff00bf090 con 0x7f20000a4170 2024-09-16T15:07:27.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7f20051a7640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fe4097870 0x7f1fe4099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe4150e6640 1 --2- 172.21.15.133:0/2599446329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 0x7fe4100a8610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe408009d90 tx=0x7fe40801c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a86508f1e0cb25e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe40f7fe640 1 -- 172.21.15.133:0/2599446329 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080a6700 con 0x7fe4100ab810 2024-09-16T15:07:27.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe40f7fe640 1 -- 172.21.15.133:0/2599446329 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4080a6860 con 0x7fe4100ab810 2024-09-16T15:07:27.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe40f7fe640 1 -- 172.21.15.133:0/2599446329 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080a6bf0 con 0x7fe4100ab810 2024-09-16T15:07:27.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7f20051a7640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fe4097870 0x7f1fe4099d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc01a060 tx=0x7f1ffc0a7000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 -- 172.21.15.133:0/2599446329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 msgr2=0x7fe4100a8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 --2- 172.21.15.133:0/2599446329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 0x7fe4100a8610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe408009d90 tx=0x7fe40801c830 comp rx=0 tx=0).stop 2024-09-16T15:07:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 -- 172.21.15.133:0/2599446329 shutdown_connections 2024-09-16T15:07:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 --2- 172.21.15.133:0/2599446329 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 0x7fe4100a8610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 --2- 172.21.15.133:0/2599446329 >> v2:172.21.15.191:3300/0 conn(0x7fe4100a9ae0 0x7fe4100a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 -- 172.21.15.133:0/2599446329 >> 172.21.15.133:0/2599446329 conn(0x7fe41006a030 msgr2=0x7fe41006bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 -- 172.21.15.133:0/2599446329 shutdown_connections 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 -- 172.21.15.133:0/2599446329 wait complete. 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 Processor -- start 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 -- start start 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4100a9ae0 0x7fe4100a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 0x7fe410147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4101365b0 0x7fe410145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe416b49640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe410144640 con 0x7fe4101365b0 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 0x7fe410147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe4150e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4101365b0 0x7fe410145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe4150e6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4101365b0 0x7fe410145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57320/0 (socket says 172.21.15.133:57320) 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.398+0000 7fe4150e6640 1 -- 172.21.15.133:0/3572347840 learned_addr learned my addr 172.21.15.133:0/3572347840 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.402+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100b28a0 con 0x7fe4100a9ae0 2024-09-16T15:07:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.402+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100a4d50 con 0x7fe4100ab810 2024-09-16T15:07:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.402+0000 7fe4148e5640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4100a9ae0 0x7fe4100a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.402+0000 7fe4150e6640 1 -- 172.21.15.133:0/3572347840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 msgr2=0x7fe410147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.402+0000 7fe4150e6640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 0x7fe410147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.402+0000 7fe4150e6640 1 -- 172.21.15.133:0/3572347840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4100a9ae0 msgr2=0x7fe4100a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.402+0000 7fe4150e6640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4100a9ae0 0x7fe4100a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.402+0000 7fe4150e6640 1 -- 172.21.15.133:0/3572347840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408009a70 con 0x7fe4101365b0 2024-09-16T15:07:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.402+0000 7fe40ffff640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 0x7fe410147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.402+0000 7fe4150e6640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4101365b0 0x7fe410145ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe40801aee0 tx=0x7fe4080195d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe40d7fa640 1 -- 172.21.15.133:0/3572347840 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408009450 con 0x7fe4101365b0 2024-09-16T15:07:27.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe40d7fa640 1 -- 172.21.15.133:0/3572347840 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4080b8430 con 0x7fe4101365b0 2024-09-16T15:07:27.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe40d7fa640 1 -- 172.21.15.133:0/3572347840 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080b8800 con 0x7fe4101365b0 2024-09-16T15:07:27.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7f1ff5ffb640 1 -- 172.21.15.133:0/2399976167 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1ff008c8e0 con 0x7f20000a4170 2024-09-16T15:07:27.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4100a4ff0 con 0x7fe4101365b0 2024-09-16T15:07:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4100b31b0 con 0x7fe4101365b0 2024-09-16T15:07:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4100672a0 con 0x7fe4101365b0 2024-09-16T15:07:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe40d7fa640 1 -- 172.21.15.133:0/3572347840 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fe4080b8960 con 0x7fe4101365b0 2024-09-16T15:07:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe40d7fa640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe40d7fa640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe40d7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe40d7fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe40d7fa640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe3f4097840 0x7fe3f4099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.406+0000 7fe40d7fa640 1 -- 172.21.15.133:0/3572347840 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7fe4080aa070 con 0x7fe4101365b0 2024-09-16T15:07:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.410+0000 7fe4148e5640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe3f4097840 0x7fe3f4099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.410+0000 7fe4148e5640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe3f4097840 0x7fe3f4099d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe404000f70 tx=0x7fe404007df0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.418+0000 7fe40d7fa640 1 -- 172.21.15.133:0/3572347840 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe408128fe0 con 0x7fe4101365b0 2024-09-16T15:07:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f596717c640 1 Processor -- start 2024-09-16T15:07:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f596717c640 1 -- start start 2024-09-16T15:07:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f596717c640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f59600a5080 0x7f59600a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f596717c640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596006abb0 con 0x7f5960141e50 2024-09-16T15:07:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f596717c640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59600a5eb0 con 0x7f596006a7d0 2024-09-16T15:07:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f596717c640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59600a9600 con 0x7f59600a5080 2024-09-16T15:07:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5960141e50 0x7f5960149b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33954/0 (socket says 172.21.15.133:33954) 2024-09-16T15:07:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595ffff640 1 -- 172.21.15.133:0/1515143071 learned_addr learned my addr 172.21.15.133:0/1515143071 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645440377 0 0) 0x7f596006abb0 con 0x7f5960141e50 2024-09-16T15:07:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f5965719640 1 --2- 172.21.15.133:0/1515143071 >> v2:172.21.15.191:3300/0 conn(0x7f59600a5080 0x7f59600a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5950003530 con 0x7f5960141e50 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4183817038 0 0) 0x7f5950003530 con 0x7f5960141e50 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 >> v2:172.21.15.191:3300/0 conn(0x7f59600a5080 msgr2=0x7f59600a8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 --2- 172.21.15.133:0/1515143071 >> v2:172.21.15.191:3300/0 conn(0x7f59600a5080 0x7f59600a8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 >> v1:172.21.15.191:6789/0 conn(0x7f596006a7d0 legacy=0x7f5960140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59600a98c0 con 0x7f5960141e50 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f594c001680 con 0x7f5960141e50 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 >> v1:172.21.15.133:6789/0 conn(0x7f5960141e50 legacy=0x7f5960149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5950003980 0x7f5950003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59500047d0 0x7f5950004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5950005190 0x7f595000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59600a9600 con 0x7f5950005190 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595000c060 con 0x7f5950003980 2024-09-16T15:07:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595000c220 con 0x7f59500047d0 2024-09-16T15:07:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595ffff640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59500047d0 0x7f5950004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f5964f18640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5950003980 0x7f5950003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f5965719640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5950005190 0x7f595000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595ffff640 1 -- 172.21.15.133:0/1515143071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5950003980 msgr2=0x7f5950003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595ffff640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5950003980 0x7f5950003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595ffff640 1 -- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5950005190 msgr2=0x7f595000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595ffff640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5950005190 0x7f595000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f595ffff640 1 -- 172.21.15.133:0/1515143071 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950003530 con 0x7f59500047d0 2024-09-16T15:07:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f5965719640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5950005190 0x7f595000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.442+0000 7f5965719640 1 -- 172.21.15.133:0/1515143071 reap_dead start 2024-09-16T15:07:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f595ffff640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59500047d0 0x7f5950004c50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f594c00c270 tx=0x7f594c01cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769f0a89ac5f6bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c005ae0 con 0x7f59500047d0 2024-09-16T15:07:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c005c60 con 0x7f59500047d0 2024-09-16T15:07:27.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f595f7fe640 1 -- 172.21.15.133:0/1515143071 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0a5880 con 0x7f59500047d0 2024-09-16T15:07:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 -- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59500047d0 msgr2=0x7f5950004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59500047d0 0x7f5950004c50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f594c00c270 tx=0x7f594c01cb80 comp rx=0 tx=0).stop 2024-09-16T15:07:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 -- 172.21.15.133:0/1515143071 shutdown_connections 2024-09-16T15:07:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 --2- 172.21.15.133:0/1515143071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59500047d0 0x7f5950004c50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 -- 172.21.15.133:0/1515143071 >> 172.21.15.133:0/1515143071 conn(0x7f596006f220 msgr2=0x7f5960069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 -- 172.21.15.133:0/1515143071 shutdown_connections 2024-09-16T15:07:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 -- 172.21.15.133:0/1515143071 wait complete. 2024-09-16T15:07:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 Processor -- start 2024-09-16T15:07:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 -- start start 2024-09-16T15:07:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59500047d0 0x7f596006c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960144d70 0x7f59600b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.446+0000 7f596717c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59600b7a80 0x7f59601dc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f596717c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596006c7d0 con 0x7f5960144d70 2024-09-16T15:07:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f596717c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59601dbf90 con 0x7f59500047d0 2024-09-16T15:07:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f596717c640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596006c550 con 0x7f59600b7a80 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f595ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960144d70 0x7f59600b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f595ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960144d70 0x7f59600b7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57342/0 (socket says 172.21.15.133:57342) 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f595ffff640 1 -- 172.21.15.133:0/3867392942 learned_addr learned my addr 172.21.15.133:0/3867392942 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f5965719640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59600b7a80 0x7f59601dc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f5965719640 1 -- 172.21.15.133:0/3867392942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59500047d0 msgr2=0x7f596006c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f5965719640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59500047d0 0x7f596006c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f5965719640 1 -- 172.21.15.133:0/3867392942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960144d70 msgr2=0x7f59600b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f5965719640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960144d70 0x7f59600b7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f5965719640 1 -- 172.21.15.133:0/3867392942 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596006ce10 con 0x7f59600b7a80 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f595ffff640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960144d70 0x7f59600b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f5965719640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59600b7a80 0x7f59601dc230 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59600b8290 tx=0x7f595800f580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f595d7fa640 1 -- 172.21.15.133:0/3867392942 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595800e040 con 0x7f59600b7a80 2024-09-16T15:07:27.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f595d7fa640 1 -- 172.21.15.133:0/3867392942 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595800f040 con 0x7f59600b7a80 2024-09-16T15:07:27.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f596717c640 1 -- 172.21.15.133:0/3867392942 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5960145170 con 0x7f59600b7a80 2024-09-16T15:07:27.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f595d7fa640 1 -- 172.21.15.133:0/3867392942 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958003bc0 con 0x7f59600b7a80 2024-09-16T15:07:27.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f596717c640 1 -- 172.21.15.133:0/3867392942 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59601dc940 con 0x7f59600b7a80 2024-09-16T15:07:27.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f594affd640 1 -- 172.21.15.133:0/3867392942 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5938005050 con 0x7f59600b7a80 2024-09-16T15:07:27.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f595d7fa640 1 -- 172.21.15.133:0/3867392942 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f5958003d20 con 0x7f59600b7a80 2024-09-16T15:07:27.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.450+0000 7f595d7fa640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:27.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f595d7fa640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:27.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f595d7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f595d7fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f595d7fa640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5944097840 0x7f5944099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f595d7fa640 1 -- 172.21.15.133:0/3867392942 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f59580c04a0 con 0x7f59600b7a80 2024-09-16T15:07:27.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f5964f18640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5944097840 0x7f5944099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f772542d640 1 Processor -- start 2024-09-16T15:07:27.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f772542d640 1 -- start start 2024-09-16T15:07:27.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f5964f18640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5944097840 0x7f5944099d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5954000c10 tx=0x7f5954009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f772542d640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f772006b3b0 0x7f77200a6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f772542d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7720143e50 0x7f7720144270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f772542d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772006c870 con 0x7f77200b70e0 2024-09-16T15:07:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f772542d640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77201447b0 con 0x7f772006b3b0 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f772542d640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772006cb30 con 0x7f7720143e50 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f7717fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7720143e50 0x7f7720144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f7717fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7720143e50 0x7f7720144270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:52974/0 (socket says 172.21.15.133:52974) 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f7717fff640 1 -- 172.21.15.133:0/3662024543 learned_addr learned my addr 172.21.15.133:0/3662024543 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f771f7fe640 1 --1- 172.21.15.133:0/3662024543 >> v1:172.21.15.133:6789/0 conn(0x7f77200b70e0 0x7f77200b7570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33970/0 (socket says 172.21.15.133:33970) 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f771effd640 1 --2- 172.21.15.133:0/3662024543 >> v2:172.21.15.191:3300/0 conn(0x7f772006b3b0 0x7f77200a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f7717fff640 1 -- 172.21.15.133:0/3662024543 >> v2:172.21.15.191:3300/0 conn(0x7f772006b3b0 msgr2=0x7f77200a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f7717fff640 1 --2- 172.21.15.133:0/3662024543 >> v2:172.21.15.191:3300/0 conn(0x7f772006b3b0 0x7f77200a6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f7717fff640 1 -- 172.21.15.133:0/3662024543 >> v1:172.21.15.133:6789/0 conn(0x7f77200b70e0 legacy=0x7f77200b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f7717fff640 1 -- 172.21.15.133:0/3662024543 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200a6010 con 0x7f7720143e50 2024-09-16T15:07:27.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.454+0000 7f771effd640 1 --2- 172.21.15.133:0/3662024543 >> v2:172.21.15.191:3300/0 conn(0x7f772006b3b0 0x7f77200a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:07:27.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.458+0000 7f7717fff640 1 --2- 172.21.15.133:0/3662024543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7720143e50 0x7f7720144270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7708009870 tx=0x7f770801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45cc3ad3af2489d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f771e7fc640 1 -- 172.21.15.133:0/3662024543 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080ab600 con 0x7f7720143e50 2024-09-16T15:07:27.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f771e7fc640 1 -- 172.21.15.133:0/3662024543 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77080a7050 con 0x7f7720143e50 2024-09-16T15:07:27.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f771e7fc640 1 -- 172.21.15.133:0/3662024543 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080ab980 con 0x7f7720143e50 2024-09-16T15:07:27.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f595d7fa640 1 -- 172.21.15.133:0/3867392942 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f595808dbb0 con 0x7f59600b7a80 2024-09-16T15:07:27.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 -- 172.21.15.133:0/3662024543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7720143e50 msgr2=0x7f7720144270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 --2- 172.21.15.133:0/3662024543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7720143e50 0x7f7720144270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7708009870 tx=0x7f770801c4b0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 -- 172.21.15.133:0/3662024543 shutdown_connections 2024-09-16T15:07:27.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 --2- 172.21.15.133:0/3662024543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7720143e50 0x7f7720144270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 --2- 172.21.15.133:0/3662024543 >> v2:172.21.15.191:3300/0 conn(0x7f772006b3b0 0x7f77200a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 -- 172.21.15.133:0/3662024543 >> 172.21.15.133:0/3662024543 conn(0x7f772006f000 msgr2=0x7f772006f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 -- 172.21.15.133:0/3662024543 shutdown_connections 2024-09-16T15:07:27.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 -- 172.21.15.133:0/3662024543 wait complete. 2024-09-16T15:07:27.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 Processor -- start 2024-09-16T15:07:27.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 -- start start 2024-09-16T15:07:27.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772006b3b0 0x7f77200b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77200b70e0 0x7f7720132a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7720143e50 0x7f7720145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7720144bd0 con 0x7f772006b3b0 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77201461f0 con 0x7f7720143e50 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f7717fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77200b70e0 0x7f7720132a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f772542d640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77201330d0 con 0x7f77200b70e0 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f771effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772006b3b0 0x7f77200b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f7717fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77200b70e0 0x7f7720132a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:52976/0 (socket says 172.21.15.133:52976) 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f7717fff640 1 -- 172.21.15.133:0/881320975 learned_addr learned my addr 172.21.15.133:0/881320975 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.462+0000 7f771f7fe640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7720143e50 0x7f7720145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f771effd640 1 -- 172.21.15.133:0/881320975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77200b70e0 msgr2=0x7f7720132a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f771effd640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77200b70e0 0x7f7720132a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f771effd640 1 -- 172.21.15.133:0/881320975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7720143e50 msgr2=0x7f7720145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f771effd640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7720143e50 0x7f7720145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f771effd640 1 -- 172.21.15.133:0/881320975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77080094d0 con 0x7f772006b3b0 2024-09-16T15:07:27.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f771effd640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772006b3b0 0x7f77200b5960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f771000e7e0 tx=0x7f771000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f77177fe640 1 -- 172.21.15.133:0/881320975 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771000c6d0 con 0x7f772006b3b0 2024-09-16T15:07:27.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f77177fe640 1 -- 172.21.15.133:0/881320975 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771000c830 con 0x7f772006b3b0 2024-09-16T15:07:27.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f77177fe640 1 -- 172.21.15.133:0/881320975 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771000cba0 con 0x7f772006b3b0 2024-09-16T15:07:27.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f772542d640 1 -- 172.21.15.133:0/881320975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7720133390 con 0x7f772006b3b0 2024-09-16T15:07:27.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f772542d640 1 -- 172.21.15.133:0/881320975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77201452b0 con 0x7f772006b3b0 2024-09-16T15:07:27.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f772542d640 1 -- 172.21.15.133:0/881320975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77200672a0 con 0x7f772006b3b0 2024-09-16T15:07:27.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f77177fe640 1 -- 172.21.15.133:0/881320975 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f771000cd00 con 0x7f772006b3b0 2024-09-16T15:07:27.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f77177fe640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:27.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f77177fe640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:27.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f77177fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f77177fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:27.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f77177fe640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7700097840 0x7f7700099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:27.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f77177fe640 1 -- 172.21.15.133:0/881320975 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f77100912b0 con 0x7f772006b3b0 2024-09-16T15:07:27.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.466+0000 7f7717fff640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7700097840 0x7f7700099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:27.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.470+0000 7f7717fff640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7700097840 0x7f7700099d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f770801c4f0 tx=0x7f77080a6000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:27.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.474+0000 7f77177fe640 1 -- 172.21.15.133:0/881320975 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f771008d560 con 0x7f772006b3b0 2024-09-16T15:07:27.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.590+0000 7f200740b640 1 -- 172.21.15.133:0/2399976167 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f200006ca50 con 0x7f20000a4170 2024-09-16T15:07:27.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f1ff5ffb640 1 -- 172.21.15.133:0/2399976167 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ff008ff00 con 0x7f20000a4170 2024-09-16T15:07:27.597 INFO:teuthology.orchestra.run.smithi133.stdout:38654705669 2024-09-16T15:07:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 10 mgrc shutdown 2024-09-16T15:07:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 -- 172.21.15.133:0/2399976167 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fe4097870 msgr2=0x7f1fe4099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fe4097870 0x7f1fe4099d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc01a060 tx=0x7f1ffc0a7000 comp rx=0 tx=0).stop 2024-09-16T15:07:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 -- 172.21.15.133:0/2399976167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20000a4170 msgr2=0x7f20000af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20000a4170 0x7f20000af8d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0000f70 tx=0x7f1ff000c890 comp rx=0 tx=0).stop 2024-09-16T15:07:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 -- 172.21.15.133:0/2399976167 shutdown_connections 2024-09-16T15:07:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20001469b0 0x7f200013dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1fe4097870 0x7f1fe4099d30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20000a4170 0x7f20000af8d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 --2- 172.21.15.133:0/2399976167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f200006b260 0x7f20000b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 -- 172.21.15.133:0/2399976167 >> 172.21.15.133:0/2399976167 conn(0x7f200006f000 msgr2=0x7f20001428f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 -- 172.21.15.133:0/2399976167 shutdown_connections 2024-09-16T15:07:27.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.594+0000 7f200740b640 1 -- 172.21.15.133:0/2399976167 wait complete. 2024-09-16T15:07:27.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.0 2024-09-16T15:07:27.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.622+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe410144780 con 0x7fe4101365b0 2024-09-16T15:07:27.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.622+0000 7fe40d7fa640 1 -- 172.21.15.133:0/3572347840 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe410144780 con 0x7fe4101365b0 2024-09-16T15:07:27.626 INFO:teuthology.orchestra.run.smithi133.stdout:38654705670 2024-09-16T15:07:27.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 10 mgrc shutdown 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe3f4097840 msgr2=0x7fe3f4099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe3f4097840 0x7fe3f4099d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe404000f70 tx=0x7fe404007df0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4101365b0 msgr2=0x7fe410145ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4101365b0 0x7fe410145ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe40801aee0 tx=0x7fe4080195d0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 shutdown_connections 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fe3f4097840 0x7fe3f4099d00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4101365b0 0x7fe410145ee0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4100ab810 0x7fe410147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 --2- 172.21.15.133:0/3572347840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4100a9ae0 0x7fe4100a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 >> 172.21.15.133:0/3572347840 conn(0x7fe41006a030 msgr2=0x7fe410137bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 shutdown_connections 2024-09-16T15:07:27.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.626+0000 7fe416b49640 1 -- 172.21.15.133:0/3572347840 wait complete. 2024-09-16T15:07:27.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 -- 172.21.15.133:0/881320975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f772006f5f0 con 0x7f772006b3b0 2024-09-16T15:07:27.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f77177fe640 1 -- 172.21.15.133:0/881320975 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7710007040 con 0x7f772006b3b0 2024-09-16T15:07:27.634 INFO:teuthology.orchestra.run.smithi133.stdout:38654705669 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 10 mgrc shutdown 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 -- 172.21.15.133:0/881320975 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7700097840 msgr2=0x7f7700099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7700097840 0x7f7700099d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f770801c4f0 tx=0x7f77080a6000 comp rx=0 tx=0).stop 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 -- 172.21.15.133:0/881320975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772006b3b0 msgr2=0x7f77200b5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772006b3b0 0x7f77200b5960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f771000e7e0 tx=0x7f771000eca0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 -- 172.21.15.133:0/881320975 shutdown_connections 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7720143e50 0x7f7720145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77200b70e0 0x7f7720132a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7700097840 0x7f7700099d00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 --2- 172.21.15.133:0/881320975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772006b3b0 0x7f77200b5960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 -- 172.21.15.133:0/881320975 >> 172.21.15.133:0/881320975 conn(0x7f772006f000 msgr2=0x7f77200a81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 -- 172.21.15.133:0/881320975 shutdown_connections 2024-09-16T15:07:27.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.630+0000 7f772542d640 1 -- 172.21.15.133:0/881320975 wait complete. 2024-09-16T15:07:27.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705670 for osd.2 2024-09-16T15:07:27.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.3 2024-09-16T15:07:27.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 -- 172.21.15.133:0/3867392942 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f59380055d0 con 0x7f59600b7a80 2024-09-16T15:07:27.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f595d7fa640 1 -- 172.21.15.133:0/3867392942 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f595808fae0 con 0x7f59600b7a80 2024-09-16T15:07:27.678 INFO:teuthology.orchestra.run.smithi133.stdout:38654705669 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 10 mgrc shutdown 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 -- 172.21.15.133:0/3867392942 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5944097840 msgr2=0x7f5944099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5944097840 0x7f5944099d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5954000c10 tx=0x7f5954009040 comp rx=0 tx=0).stop 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 -- 172.21.15.133:0/3867392942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59600b7a80 msgr2=0x7f59601dc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59600b7a80 0x7f59601dc230 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59600b8290 tx=0x7f595800f580 comp rx=0 tx=0).stop 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 -- 172.21.15.133:0/3867392942 shutdown_connections 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59600b7a80 0x7f59601dc230 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5944097840 0x7f5944099d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5960144d70 0x7f59600b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 --2- 172.21.15.133:0/3867392942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59500047d0 0x7f596006c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:27.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 -- 172.21.15.133:0/3867392942 >> 172.21.15.133:0/3867392942 conn(0x7f596006f220 msgr2=0x7f59600a6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:27.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 -- 172.21.15.133:0/3867392942 shutdown_connections 2024-09-16T15:07:27.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:27.674+0000 7f594affd640 1 -- 172.21.15.133:0/3867392942 wait complete. 2024-09-16T15:07:27.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.1 2024-09-16T15:07:28.617 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T15:07:28.654 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T15:07:28.654 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T15:07:28.695 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T15:07:28.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2974207640 1 Processor -- start 2024-09-16T15:07:28.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2974207640 1 -- start start 2024-09-16T15:07:28.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2974207640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f296c06a0c0 0x7f296c0a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2974207640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296c06a4a0 con 0x7f296c0a37e0 2024-09-16T15:07:28.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2974207640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296c0a6c50 con 0x7f296c06cfd0 2024-09-16T15:07:28.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2974207640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296c144610 con 0x7f296c06a0c0 2024-09-16T15:07:28.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f29717a2640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f296c0a37e0 0x7f296c143f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33984/0 (socket says 172.21.15.133:33984) 2024-09-16T15:07:28.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f29717a2640 1 -- 172.21.15.133:0/3492991960 learned_addr learned my addr 172.21.15.133:0/3492991960 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:28.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2971fa3640 1 --2- 172.21.15.133:0/3492991960 >> v2:172.21.15.191:3300/0 conn(0x7f296c06a0c0 0x7f296c0a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106715762 0 0) 0x7f296c06a4a0 con 0x7f296c0a37e0 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2958003530 con 0x7f296c0a37e0 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263370724 0 0) 0x7f2958003530 con 0x7f296c0a37e0 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 >> v2:172.21.15.191:3300/0 conn(0x7f296c06a0c0 msgr2=0x7f296c0a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 --2- 172.21.15.133:0/3492991960 >> v2:172.21.15.191:3300/0 conn(0x7f296c06a0c0 0x7f296c0a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 >> v1:172.21.15.191:6789/0 conn(0x7f296c06cfd0 legacy=0x7f296c14dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0a6ef0 con 0x7f296c0a37e0 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f295c001680 con 0x7f296c0a37e0 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 >> v1:172.21.15.133:6789/0 conn(0x7f296c0a37e0 legacy=0x7f296c143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 --2- 172.21.15.133:0/3492991960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2958003980 0x7f2958003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 --2- 172.21.15.133:0/3492991960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 0x7f2958004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 --2- 172.21.15.133:0/3492991960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2958005190 0x7f295800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c144610 con 0x7f2958005190 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295800c130 con 0x7f2958003980 2024-09-16T15:07:28.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295800c2f0 con 0x7f29580047d0 2024-09-16T15:07:28.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.746+0000 7f29717a2640 1 --2- 172.21.15.133:0/3492991960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 0x7f2958004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f29727a4640 1 --2- 172.21.15.133:0/3492991960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2958005190 0x7f295800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f29717a2640 1 -- 172.21.15.133:0/3492991960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2958003980 msgr2=0x7f2958003da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T15:07:28.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f29717a2640 1 --2- 172.21.15.133:0/3492991960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2958003980 0x7f2958003da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f29717a2640 1 -- 172.21.15.133:0/3492991960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2958005190 msgr2=0x7f295800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:28.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f29717a2640 1 --2- 172.21.15.133:0/3492991960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2958005190 0x7f295800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f29717a2640 1 -- 172.21.15.133:0/3492991960 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958003530 con 0x7f29580047d0 2024-09-16T15:07:28.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f29727a4640 1 -- 172.21.15.133:0/3492991960 reap_dead start 2024-09-16T15:07:28.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f29717a2640 1 --2- 172.21.15.133:0/3492991960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 0x7f2958004c50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f295c00c240 tx=0x7f295c01f3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee6f6347ec16035 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c005ae0 con 0x7f29580047d0 2024-09-16T15:07:28.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c0ab070 con 0x7f29580047d0 2024-09-16T15:07:28.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f2970fa1640 1 -- 172.21.15.133:0/3492991960 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c007070 con 0x7f29580047d0 2024-09-16T15:07:28.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f2974207640 1 -- 172.21.15.133:0/3492991960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 msgr2=0x7f2958004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f2974207640 1 --2- 172.21.15.133:0/3492991960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 0x7f2958004c50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f295c00c240 tx=0x7f295c01f3b0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f2974207640 1 -- 172.21.15.133:0/3492991960 shutdown_connections 2024-09-16T15:07:28.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f2974207640 1 --2- 172.21.15.133:0/3492991960 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 0x7f2958004c50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f2974207640 1 -- 172.21.15.133:0/3492991960 >> 172.21.15.133:0/3492991960 conn(0x7f296c0a51b0 msgr2=0x7f296c0a55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:28.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f2974207640 1 -- 172.21.15.133:0/3492991960 shutdown_connections 2024-09-16T15:07:28.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.750+0000 7f2974207640 1 -- 172.21.15.133:0/3492991960 wait complete. 2024-09-16T15:07:28.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2974207640 1 Processor -- start 2024-09-16T15:07:28.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2974207640 1 -- start start 2024-09-16T15:07:28.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2974207640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 0x7f296c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2974207640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f296c071080 0x7f296c071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2974207640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f296c1d8c10 0x7f296c1d9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2974207640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c071e50 con 0x7f296c071080 2024-09-16T15:07:28.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2974207640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1d8970 con 0x7f296c1d8c10 2024-09-16T15:07:28.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2974207640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0a1490 con 0x7f29580047d0 2024-09-16T15:07:28.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29717a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f296c071080 0x7f296c071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29717a2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f296c071080 0x7f296c071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57364/0 (socket says 172.21.15.133:57364) 2024-09-16T15:07:28.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29717a2640 1 -- 172.21.15.133:0/3874935371 learned_addr learned my addr 172.21.15.133:0/3874935371 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:28.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2971fa3640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 0x7f296c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29727a4640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f296c1d8c10 0x7f296c1d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29717a2640 1 -- 172.21.15.133:0/3874935371 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 msgr2=0x7f296c06fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29717a2640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 0x7f296c06fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29717a2640 1 -- 172.21.15.133:0/3874935371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f296c1d8c10 msgr2=0x7f296c1d9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29717a2640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f296c1d8c10 0x7f296c1d9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29717a2640 1 -- 172.21.15.133:0/3874935371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c070810 con 0x7f296c071080 2024-09-16T15:07:28.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2971fa3640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 0x7f296c06fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:28.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29717a2640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f296c071080 0x7f296c071460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f295c01f1e0 tx=0x7f2958005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29627fc640 1 -- 172.21.15.133:0/3874935371 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c001640 con 0x7f296c071080 2024-09-16T15:07:28.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29627fc640 1 -- 172.21.15.133:0/3874935371 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c007070 con 0x7f296c071080 2024-09-16T15:07:28.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2974207640 1 -- 172.21.15.133:0/3874935371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c0719a0 con 0x7f296c071080 2024-09-16T15:07:28.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f29627fc640 1 -- 172.21.15.133:0/3874935371 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c007400 con 0x7f296c071080 2024-09-16T15:07:28.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.754+0000 7f2974207640 1 -- 172.21.15.133:0/3874935371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c0a2280 con 0x7f296c071080 2024-09-16T15:07:28.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.758+0000 7f2974207640 1 -- 172.21.15.133:0/3874935371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c071fc0 con 0x7f296c071080 2024-09-16T15:07:28.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.758+0000 7f29627fc640 1 -- 172.21.15.133:0/3874935371 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f295c0b9030 con 0x7f296c071080 2024-09-16T15:07:28.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.758+0000 7f29627fc640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:28.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.758+0000 7f29627fc640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:28.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.758+0000 7f29627fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:28.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.758+0000 7f29627fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:28.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.758+0000 7f29627fc640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 0x7f2950099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.758+0000 7f29627fc640 1 -- 172.21.15.133:0/3874935371 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f295c0b7070 con 0x7f296c071080 2024-09-16T15:07:28.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.758+0000 7f2971fa3640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 0x7f2950099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.762+0000 7f2971fa3640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 0x7f2950099d00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f296c068de0 tx=0x7f2968009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.766+0000 7f29627fc640 1 -- 172.21.15.133:0/3874935371 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f295c12a5e0 con 0x7f296c071080 2024-09-16T15:07:28.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd6652c640 1 Processor -- start 2024-09-16T15:07:28.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd6652c640 1 -- start start 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd6652c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdd600660e0 0x7fdd6006b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd6652c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 0x7fdd6014dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd6652c640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd600a6ce0 con 0x7fdd60065d00 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd6652c640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd600a6f20 con 0x7fdd600660e0 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd6652c640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6006b180 con 0x7fdd600a74a0 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd5f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdd600660e0 0x7fdd6006b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd5f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdd600660e0 0x7fdd6006b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57378/0 (socket says 172.21.15.133:57378) 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd5f7fe640 1 -- 172.21.15.133:0/2408684088 learned_addr learned my addr 172.21.15.133:0/2408684088 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd64ac9640 1 --2- 172.21.15.133:0/2408684088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 0x7fdd6014dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd5f7fe640 1 -- 172.21.15.133:0/2408684088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 msgr2=0x7fdd6014dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd5f7fe640 1 --2- 172.21.15.133:0/2408684088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 0x7fdd6014dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd5f7fe640 1 -- 172.21.15.133:0/2408684088 >> v1:172.21.15.191:6789/0 conn(0x7fdd60065d00 legacy=0x7fdd600a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.774+0000 7fdd5f7fe640 1 -- 172.21.15.133:0/2408684088 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6006b440 con 0x7fdd600660e0 2024-09-16T15:07:28.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd5f7fe640 1 --2- 172.21.15.133:0/2408684088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd600660e0 0x7fdd6006b890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009870 tx=0x7fdd5401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e765063e15694769 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd5effd640 1 -- 172.21.15.133:0/2408684088 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd540ab640 con 0x7fdd600660e0 2024-09-16T15:07:28.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd5effd640 1 -- 172.21.15.133:0/2408684088 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd540a7070 con 0x7fdd600660e0 2024-09-16T15:07:28.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd5effd640 1 -- 172.21.15.133:0/2408684088 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd540ab9c0 con 0x7fdd600660e0 2024-09-16T15:07:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd6652c640 1 -- 172.21.15.133:0/2408684088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd600660e0 msgr2=0x7fdd6006b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd6652c640 1 --2- 172.21.15.133:0/2408684088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd600660e0 0x7fdd6006b890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009870 tx=0x7fdd5401c420 comp rx=0 tx=0).stop 2024-09-16T15:07:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd6652c640 1 -- 172.21.15.133:0/2408684088 shutdown_connections 2024-09-16T15:07:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd6652c640 1 --2- 172.21.15.133:0/2408684088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 0x7fdd6014dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd6652c640 1 --2- 172.21.15.133:0/2408684088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd600660e0 0x7fdd6006b890 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd6652c640 1 -- 172.21.15.133:0/2408684088 >> 172.21.15.133:0/2408684088 conn(0x7fdd6006a030 msgr2=0x7fdd6006a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd6652c640 1 -- 172.21.15.133:0/2408684088 shutdown_connections 2024-09-16T15:07:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd6652c640 1 -- 172.21.15.133:0/2408684088 wait complete. 2024-09-16T15:07:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd6652c640 1 Processor -- start 2024-09-16T15:07:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.778+0000 7fdd6652c640 1 -- start start 2024-09-16T15:07:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd6652c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd60065d00 0x7fdd60148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd6652c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 0x7fdd6013b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd6652c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6013bb30 0x7fdd600a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd6652c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6009df40 con 0x7fdd6013bb30 2024-09-16T15:07:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd6652c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6009e190 con 0x7fdd60065d00 2024-09-16T15:07:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd6652c640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601482a0 con 0x7fdd600a74a0 2024-09-16T15:07:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 0x7fdd6013b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 0x7fdd6013b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53010/0 (socket says 172.21.15.133:53010) 2024-09-16T15:07:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5f7fe640 1 -- 172.21.15.133:0/1767887027 learned_addr learned my addr 172.21.15.133:0/1767887027 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:28.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd64ac9640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6013bb30 0x7fdd600a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5ffff640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd60065d00 0x7fdd60148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5f7fe640 1 -- 172.21.15.133:0/1767887027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd60065d00 msgr2=0x7fdd60148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5f7fe640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd60065d00 0x7fdd60148a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5f7fe640 1 -- 172.21.15.133:0/1767887027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6013bb30 msgr2=0x7fdd600a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5f7fe640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6013bb30 0x7fdd600a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5f7fe640 1 -- 172.21.15.133:0/1767887027 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd540094d0 con 0x7fdd600a74a0 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd64ac9640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6013bb30 0x7fdd600a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5f7fe640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 0x7fdd6013b5f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdd540ad040 tx=0x7fdd5401ac60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5cff9640 1 -- 172.21.15.133:0/1767887027 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd540abea0 con 0x7fdd600a74a0 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5cff9640 1 -- 172.21.15.133:0/1767887027 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd540be3f0 con 0x7fdd600a74a0 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd6652c640 1 -- 172.21.15.133:0/1767887027 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd60148580 con 0x7fdd600a74a0 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd6652c640 1 -- 172.21.15.133:0/1767887027 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6009f960 con 0x7fdd600a74a0 2024-09-16T15:07:28.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5cff9640 1 -- 172.21.15.133:0/1767887027 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd540be7a0 con 0x7fdd600a74a0 2024-09-16T15:07:28.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7d00b69640 1 Processor -- start 2024-09-16T15:07:28.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd6652c640 1 -- 172.21.15.133:0/1767887027 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd40005050 con 0x7fdd600a74a0 2024-09-16T15:07:28.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7d00b69640 1 -- start start 2024-09-16T15:07:28.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7d00b69640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cfc06a0c0 0x7f7cfc09ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7d00b69640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7cfc0a3640 0x7f7cfc14b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cfa59c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cfc06a0c0 0x7f7cfc09ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf9d9b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7cfc09f210 0x7f7cfc06cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33994/0 (socket says 172.21.15.133:33994) 2024-09-16T15:07:28.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5cff9640 1 -- 172.21.15.133:0/1767887027 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fdd540b43f0 con 0x7fdd600a74a0 2024-09-16T15:07:28.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7fdd5cff9640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:28.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf9d9b640 1 -- 172.21.15.133:0/193969125 learned_addr learned my addr 172.21.15.133:0/193969125 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:28.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7d00b69640 1 -- 172.21.15.133:0/193969125 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc06a4a0 con 0x7f7cfc09f210 2024-09-16T15:07:28.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7d00b69640 1 -- 172.21.15.133:0/193969125 --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc0a2960 con 0x7f7cfc0a3640 2024-09-16T15:07:28.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cfad9d640 1 --2- 172.21.15.133:0/193969125 >> v2:172.21.15.191:3300/0 conn(0x7f7cfc0a3640 0x7f7cfc14b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7d00b69640 1 -- 172.21.15.133:0/193969125 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc09fa90 con 0x7f7cfc06a0c0 2024-09-16T15:07:28.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485491520 0 0) 0x7f7cfc06a4a0 con 0x7f7cfc09f210 2024-09-16T15:07:28.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce4003530 con 0x7f7cfc09f210 2024-09-16T15:07:28.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3680149691 0 0) 0x7f7ce4003530 con 0x7f7cfc09f210 2024-09-16T15:07:28.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cfc06a0c0 msgr2=0x7f7cfc09ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:28.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf959a640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cfc06a0c0 0x7f7cfc09ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 >> v2:172.21.15.191:3300/0 conn(0x7f7cfc0a3640 msgr2=0x7f7cfc14b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:28.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf959a640 1 --2- 172.21.15.133:0/193969125 >> v2:172.21.15.191:3300/0 conn(0x7f7cfc0a3640 0x7f7cfc14b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc09f860 con 0x7f7cfc09f210 2024-09-16T15:07:28.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.782+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f7cf0001680 con 0x7f7cfc09f210 2024-09-16T15:07:28.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.786+0000 7fdd5cff9640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:28.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.786+0000 7fdd5cff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:28.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.786+0000 7fdd5cff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:28.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.786+0000 7fdd5cff9640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd4c0978c0 0x7fdd4c099d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.786+0000 7fdd5cff9640 1 -- 172.21.15.133:0/1767887027 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7fdd5415a650 con 0x7fdd600a74a0 2024-09-16T15:07:28.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.786+0000 7fdd5ffff640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd4c0978c0 0x7fdd4c099d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.786+0000 7fdd5ffff640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd4c0978c0 0x7fdd4c099d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000f70 tx=0x7fdd50009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfa59c640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cfc06a0c0 0x7f7cfc09ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfad9d640 1 --2- 172.21.15.133:0/193969125 >> v2:172.21.15.191:3300/0 conn(0x7f7cfc0a3640 0x7f7cfc14b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 >> v1:172.21.15.133:6789/0 conn(0x7f7cfc09f210 legacy=0x7f7cfc06cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf959a640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4003980 0x7f7ce4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf959a640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cfc06a0c0 0x7f7ce40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf9d9b640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cfc06a0c0 0x7f7ce40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfa59c640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4003980 0x7f7ce4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7fdd5cff9640 1 -- 172.21.15.133:0/1767887027 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd54127cc0 con 0x7fdd600a74a0 2024-09-16T15:07:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf959a640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ce4004d10 0x7f7ce4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0a2960 con 0x7f7ce4003980 2024-09-16T15:07:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc09fa90 con 0x7f7ce4004d10 2024-09-16T15:07:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce400bb40 con 0x7f7cfc06a0c0 2024-09-16T15:07:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfa59c640 1 -- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cfc06a0c0 msgr2=0x7f7ce40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfad9d640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ce4004d10 0x7f7ce4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfa59c640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cfc06a0c0 0x7f7ce40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfa59c640 1 -- 172.21.15.133:0/193969125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ce4004d10 msgr2=0x7f7ce4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfa59c640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ce4004d10 0x7f7ce4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfa59c640 1 -- 172.21.15.133:0/193969125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce4003530 con 0x7f7ce4003980 2024-09-16T15:07:28.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfad9d640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ce4004d10 0x7f7ce4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:28.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfa59c640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4003980 0x7f7ce4003d60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00aaa0 tx=0x7f7cec01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fc2a0b9c6b614b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0ab600 con 0x7f7ce4003980 2024-09-16T15:07:28.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec0ab760 con 0x7f7ce4003980 2024-09-16T15:07:28.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cf959a640 1 -- 172.21.15.133:0/193969125 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0abb10 con 0x7f7ce4003980 2024-09-16T15:07:28.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7d00b69640 1 -- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4003980 msgr2=0x7f7ce4003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7d00b69640 1 --2- 172.21.15.133:0/193969125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ce4003980 0x7f7ce4003d60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00aaa0 tx=0x7f7cec01c530 comp rx=0 tx=0).stop 2024-09-16T15:07:28.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7cfad9d640 1 -- 172.21.15.133:0/193969125 reap_dead start 2024-09-16T15:07:28.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7d00b69640 1 -- 172.21.15.133:0/193969125 shutdown_connections 2024-09-16T15:07:28.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7d00b69640 1 -- 172.21.15.133:0/193969125 >> 172.21.15.133:0/193969125 conn(0x7f7cfc0a51b0 msgr2=0x7f7cfc0a55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:28.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7d00b69640 1 -- 172.21.15.133:0/193969125 shutdown_connections 2024-09-16T15:07:28.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.790+0000 7f7d00b69640 1 -- 172.21.15.133:0/193969125 wait complete. 2024-09-16T15:07:28.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 Processor -- start 2024-09-16T15:07:28.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 -- start start 2024-09-16T15:07:28.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ce4003980 0x7f7cfc14ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cfc1dcb60 0x7f7cfc09ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cfc142650 0x7f7cfc09ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc09c5c0 con 0x7f7cfc142650 2024-09-16T15:07:28.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1dd030 con 0x7f7cfc1dcb60 2024-09-16T15:07:28.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1dd2d0 con 0x7f7ce4003980 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfa59c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ce4003980 0x7f7cfc14ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cfc142650 0x7f7cfc09ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cfc142650 0x7f7cfc09ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57396/0 (socket says 172.21.15.133:57396) 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfa59c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ce4003980 0x7f7cfc14ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53026/0 (socket says 172.21.15.133:53026) 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfad9d640 1 -- 172.21.15.133:0/1460939647 learned_addr learned my addr 172.21.15.133:0/1460939647 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cf9d9b640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cfc1dcb60 0x7f7cfc09ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfad9d640 1 -- 172.21.15.133:0/1460939647 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ce4003980 msgr2=0x7f7cfc14ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfad9d640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ce4003980 0x7f7cfc14ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfad9d640 1 -- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cfc1dcb60 msgr2=0x7f7cfc09ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfad9d640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cfc1dcb60 0x7f7cfc09ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfad9d640 1 -- 172.21.15.133:0/1460939647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc09cf50 con 0x7f7cfc142650 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cf9d9b640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cfc1dcb60 0x7f7cfc09ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:28.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfa59c640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ce4003980 0x7f7cfc14ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:28.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7cfad9d640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cfc142650 0x7f7cfc09ace0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0a3a20 tx=0x7f7cf400b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7ceaffd640 1 -- 172.21.15.133:0/1460939647 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf40097c0 con 0x7f7cfc142650 2024-09-16T15:07:28.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7ceaffd640 1 -- 172.21.15.133:0/1460939647 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf4009920 con 0x7f7cfc142650 2024-09-16T15:07:28.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7ceaffd640 1 -- 172.21.15.133:0/1460939647 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4009cd0 con 0x7f7cfc142650 2024-09-16T15:07:28.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 -- 172.21.15.133:0/1460939647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc071200 con 0x7f7cfc142650 2024-09-16T15:07:28.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 -- 172.21.15.133:0/1460939647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc09b220 con 0x7f7cfc142650 2024-09-16T15:07:28.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.794+0000 7f7d00b69640 1 -- 172.21.15.133:0/1460939647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc09c730 con 0x7f7cfc142650 2024-09-16T15:07:28.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.798+0000 7f7ceaffd640 1 -- 172.21.15.133:0/1460939647 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f7cf401e3b0 con 0x7f7cfc142650 2024-09-16T15:07:28.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.798+0000 7f7ceaffd640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:28.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.798+0000 7f7ceaffd640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:28.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.798+0000 7f7ceaffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:28.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.798+0000 7f7ceaffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:28.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.798+0000 7f7ceaffd640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7cdc097840 0x7f7cdc099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.798+0000 7f7ceaffd640 1 -- 172.21.15.133:0/1460939647 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f7cf40c0570 con 0x7f7cfc142650 2024-09-16T15:07:28.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.798+0000 7f7cfa59c640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7cdc097840 0x7f7cdc099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.798+0000 7f7cfa59c640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7cdc097840 0x7f7cdc099d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0af000 tx=0x7f7cec0046d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.802+0000 7f7ceaffd640 1 -- 172.21.15.133:0/1460939647 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7cf408dd10 con 0x7f7cfc142650 2024-09-16T15:07:28.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f80564e7640 1 Processor -- start 2024-09-16T15:07:28.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f80564e7640 1 -- start start 2024-09-16T15:07:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f80564e7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f80500a3660 0x7f8050141d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f80564e7640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805009fbd0 con 0x7f8050069e60 2024-09-16T15:07:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f80564e7640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805006a540 con 0x7f80500a3d40 2024-09-16T15:07:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f80564e7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80500a4120 con 0x7f80500a3660 2024-09-16T15:07:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f8054a84640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f80500a3660 0x7f8050141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f804f7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8050069e60 0x7f8050141680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34000/0 (socket says 172.21.15.133:34000) 2024-09-16T15:07:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f8054a84640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f80500a3660 0x7f8050141d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57404/0 (socket says 172.21.15.133:57404) 2024-09-16T15:07:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f804f7fe640 1 -- 172.21.15.133:0/1564180331 learned_addr learned my addr 172.21.15.133:0/1564180331 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f8054a84640 1 -- 172.21.15.133:0/1564180331 >> v1:172.21.15.191:6789/0 conn(0x7f80500a3d40 legacy=0x7f805006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f8054a84640 1 -- 172.21.15.133:0/1564180331 >> v1:172.21.15.133:6789/0 conn(0x7f8050069e60 legacy=0x7f8050141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f804effd640 1 -- 172.21.15.133:0/1564180331 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231120847 0 0) 0x7f805009fbd0 con 0x7f8050069e60 2024-09-16T15:07:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.842+0000 7f8054a84640 1 -- 172.21.15.133:0/1564180331 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80500a4260 con 0x7f80500a3660 2024-09-16T15:07:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f8054a84640 1 --2- 172.21.15.133:0/1564180331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80500a3660 0x7f8050141d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8048009dc0 tx=0x7f804801c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecf20afa59d5a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f804effd640 1 -- 172.21.15.133:0/1564180331 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80480a57e0 con 0x7f80500a3660 2024-09-16T15:07:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f804effd640 1 -- 172.21.15.133:0/1564180331 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80480a5940 con 0x7f80500a3660 2024-09-16T15:07:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f804effd640 1 -- 172.21.15.133:0/1564180331 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80480a5cd0 con 0x7f80500a3660 2024-09-16T15:07:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f80564e7640 1 -- 172.21.15.133:0/1564180331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80500a3660 msgr2=0x7f8050141d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f80564e7640 1 --2- 172.21.15.133:0/1564180331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80500a3660 0x7f8050141d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8048009dc0 tx=0x7f804801c830 comp rx=0 tx=0).stop 2024-09-16T15:07:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f80564e7640 1 -- 172.21.15.133:0/1564180331 shutdown_connections 2024-09-16T15:07:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f80564e7640 1 --2- 172.21.15.133:0/1564180331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80500a3660 0x7f8050141d90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f80564e7640 1 -- 172.21.15.133:0/1564180331 >> 172.21.15.133:0/1564180331 conn(0x7f80500a4f70 msgr2=0x7f80500a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f80564e7640 1 -- 172.21.15.133:0/1564180331 shutdown_connections 2024-09-16T15:07:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f80564e7640 1 -- 172.21.15.133:0/1564180331 wait complete. 2024-09-16T15:07:28.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.846+0000 7f80564e7640 1 Processor -- start 2024-09-16T15:07:28.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f80564e7640 1 -- start start 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f80564e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8050069e60 0x7f80500a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f80564e7640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80500a3660 0x7f8050138920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f80564e7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80500a3d40 0x7f8050138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f80564e7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805009e770 con 0x7f8050069e60 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f80564e7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80500a67c0 con 0x7f80500a3d40 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f80564e7640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80500a6a60 con 0x7f80500a3660 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8050069e60 0x7f80500a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8050069e60 0x7f80500a7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57418/0 (socket says 172.21.15.133:57418) 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804ffff640 1 -- 172.21.15.133:0/3146445959 learned_addr learned my addr 172.21.15.133:0/3146445959 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804f7fe640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80500a3660 0x7f8050138920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f8054a84640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80500a3d40 0x7f8050138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804ffff640 1 -- 172.21.15.133:0/3146445959 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80500a3660 msgr2=0x7f8050138920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804ffff640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80500a3660 0x7f8050138920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804ffff640 1 -- 172.21.15.133:0/3146445959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80500a3d40 msgr2=0x7f8050138e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804ffff640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80500a3d40 0x7f8050138e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804ffff640 1 -- 172.21.15.133:0/3146445959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048009a70 con 0x7f8050069e60 2024-09-16T15:07:28.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804ffff640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8050069e60 0x7f80500a7f20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f804000c500 tx=0x7f804000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804cff9640 1 -- 172.21.15.133:0/3146445959 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040005220 con 0x7f8050069e60 2024-09-16T15:07:28.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804cff9640 1 -- 172.21.15.133:0/3146445959 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8040005380 con 0x7f8050069e60 2024-09-16T15:07:28.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f80564e7640 1 -- 172.21.15.133:0/3146445959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80501393a0 con 0x7f8050069e60 2024-09-16T15:07:28.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f804cff9640 1 -- 172.21.15.133:0/3146445959 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040007270 con 0x7f8050069e60 2024-09-16T15:07:28.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f80564e7640 1 -- 172.21.15.133:0/3146445959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805009ce50 con 0x7f8050069e60 2024-09-16T15:07:28.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.850+0000 7f80564e7640 1 -- 172.21.15.133:0/3146445959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f805006cc40 con 0x7f8050069e60 2024-09-16T15:07:28.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.854+0000 7f804cff9640 1 -- 172.21.15.133:0/3146445959 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f80400073d0 con 0x7f8050069e60 2024-09-16T15:07:28.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.854+0000 7f804cff9640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:28.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.854+0000 7f804cff9640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:28.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.854+0000 7f804cff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:28.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.854+0000 7f804cff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:28.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.854+0000 7f804cff9640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f803c097840 0x7f803c099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:28.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.854+0000 7f804cff9640 1 -- 172.21.15.133:0/3146445959 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f80400c02b0 con 0x7f8050069e60 2024-09-16T15:07:28.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.854+0000 7f804f7fe640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f803c097840 0x7f803c099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:28.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.854+0000 7f804f7fe640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f803c097840 0x7f803c099d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8044007bc0 tx=0x7f8044007b10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:28.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.862+0000 7f804cff9640 1 -- 172.21.15.133:0/3146445959 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f804008dae0 con 0x7f8050069e60 2024-09-16T15:07:28.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.958+0000 7f2974207640 1 -- 172.21.15.133:0/3874935371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f296c0a6500 con 0x7f296c071080 2024-09-16T15:07:28.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.958+0000 7f29627fc640 1 -- 172.21.15.133:0/3874935371 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f295c0c1020 con 0x7f296c071080 2024-09-16T15:07:28.962 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 -- 172.21.15.133:0/1460939647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7cfc06cbc0 con 0x7f7cfc142650 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 10 mgrc shutdown 2024-09-16T15:07:28.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 -- 172.21.15.133:0/3874935371 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 msgr2=0x7f2950099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 0x7f2950099d00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f296c068de0 tx=0x7f2968009190 comp rx=0 tx=0).stop 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 -- 172.21.15.133:0/3874935371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f296c071080 msgr2=0x7f296c071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7ceaffd640 1 -- 172.21.15.133:0/1460939647 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7cf4007690 con 0x7f7cfc142650 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f296c071080 0x7f296c071460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f295c01f1e0 tx=0x7f2958005800 comp rx=0 tx=0).stop 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 -- 172.21.15.133:0/3874935371 shutdown_connections 2024-09-16T15:07:28.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f296c1d8c10 0x7f296c1d9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 0x7f2950099d00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f296c071080 0x7f296c071460 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 --2- 172.21.15.133:0/3874935371 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29580047d0 0x7f296c06fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi133.stdout:38654705672 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 -- 172.21.15.133:0/3874935371 >> 172.21.15.133:0/3874935371 conn(0x7f296c0a51b0 msgr2=0x7f296c06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:28.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 -- 172.21.15.133:0/3874935371 shutdown_connections 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f2974207640 1 -- 172.21.15.133:0/3874935371 wait complete. 2024-09-16T15:07:28.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 10 mgrc shutdown 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 -- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7cdc097840 msgr2=0x7f7cdc099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7cdc097840 0x7f7cdc099d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7cec0af000 tx=0x7f7cec0046d0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 -- 172.21.15.133:0/1460939647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cfc142650 msgr2=0x7f7cfc09ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cfc142650 0x7f7cfc09ace0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0a3a20 tx=0x7f7cf400b7a0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 -- 172.21.15.133:0/1460939647 shutdown_connections 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f7cdc097840 0x7f7cdc099d00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cfc142650 0x7f7cfc09ace0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cfc1dcb60 0x7f7cfc09ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 --2- 172.21.15.133:0/1460939647 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ce4003980 0x7f7cfc14ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 -- 172.21.15.133:0/1460939647 >> 172.21.15.133:0/1460939647 conn(0x7f7cfc0a51b0 msgr2=0x7f7cfc0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 -- 172.21.15.133:0/1460939647 shutdown_connections 2024-09-16T15:07:28.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:28.962+0000 7f7d00b69640 1 -- 172.21.15.133:0/1460939647 wait complete. 2024-09-16T15:07:28.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705671 for osd.0 2024-09-16T15:07:28.985 DEBUG:teuthology.parallel:result is None 2024-09-16T15:07:28.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705672 for osd.2 2024-09-16T15:07:28.986 DEBUG:teuthology.parallel:result is None 2024-09-16T15:07:29.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.022+0000 7fdd6652c640 1 -- 172.21.15.133:0/1767887027 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdd40005300 con 0x7fdd600a74a0 2024-09-16T15:07:29.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.022+0000 7fdd5cff9640 1 -- 172.21.15.133:0/1767887027 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdd54129bf0 con 0x7fdd600a74a0 2024-09-16T15:07:29.028 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-09-16T15:07:29.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 10 mgrc shutdown 2024-09-16T15:07:29.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 -- 172.21.15.133:0/1767887027 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd4c0978c0 msgr2=0x7fdd4c099d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd4c0978c0 0x7fdd4c099d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000f70 tx=0x7fdd50009040 comp rx=0 tx=0).stop 2024-09-16T15:07:29.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 -- 172.21.15.133:0/1767887027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 msgr2=0x7fdd6013b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 0x7fdd6013b5f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdd540ad040 tx=0x7fdd5401ac60 comp rx=0 tx=0).stop 2024-09-16T15:07:29.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 -- 172.21.15.133:0/1767887027 shutdown_connections 2024-09-16T15:07:29.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdd4c0978c0 0x7fdd4c099d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd6013bb30 0x7fdd600a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd600a74a0 0x7fdd6013b5f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 --2- 172.21.15.133:0/1767887027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd60065d00 0x7fdd60148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 -- 172.21.15.133:0/1767887027 >> 172.21.15.133:0/1767887027 conn(0x7fdd6006a030 msgr2=0x7fdd600a2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 -- 172.21.15.133:0/1767887027 shutdown_connections 2024-09-16T15:07:29.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.026+0000 7fdd6652c640 1 -- 172.21.15.133:0/1767887027 wait complete. 2024-09-16T15:07:29.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705671 for osd.3 2024-09-16T15:07:29.044 DEBUG:teuthology.parallel:result is None 2024-09-16T15:07:29.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.082+0000 7f80564e7640 1 -- 172.21.15.133:0/3146445959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f805006b110 con 0x7f8050069e60 2024-09-16T15:07:29.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.082+0000 7f804cff9640 1 -- 172.21.15.133:0/3146445959 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8040091100 con 0x7f8050069e60 2024-09-16T15:07:29.088 INFO:teuthology.orchestra.run.smithi133.stdout:38654705671 2024-09-16T15:07:29.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 10 mgrc shutdown 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 -- 172.21.15.133:0/3146445959 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f803c097840 msgr2=0x7f803c099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f803c097840 0x7f803c099d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8044007bc0 tx=0x7f8044007b10 comp rx=0 tx=0).stop 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 -- 172.21.15.133:0/3146445959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8050069e60 msgr2=0x7f80500a7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8050069e60 0x7f80500a7f20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f804000c500 tx=0x7f804000c9c0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 -- 172.21.15.133:0/3146445959 shutdown_connections 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80500a3d40 0x7f8050138e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80500a3660 0x7f8050138920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f803c097840 0x7f803c099d00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 --2- 172.21.15.133:0/3146445959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8050069e60 0x7f80500a7f20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 -- 172.21.15.133:0/3146445959 >> 172.21.15.133:0/3146445959 conn(0x7f80500a4f70 msgr2=0x7f8050143a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 -- 172.21.15.133:0/3146445959 shutdown_connections 2024-09-16T15:07:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.086+0000 7f80564e7640 1 -- 172.21.15.133:0/3146445959 wait complete. 2024-09-16T15:07:29.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705671 for osd.1 2024-09-16T15:07:29.105 DEBUG:teuthology.parallel:result is None 2024-09-16T15:07:29.105 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:07:29.105 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:07:29.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe49268640 1 Processor -- start 2024-09-16T15:07:29.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe49268640 1 -- start start 2024-09-16T15:07:29.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe49268640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbe440a2cf0 0x7fbe440a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe49268640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbe4409e490 0x7fbe4406afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe49268640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 0x7fbe44144a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe49268640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4406a8f0 con 0x7fbe440a2cf0 2024-09-16T15:07:29.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe49268640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe440a5bd0 con 0x7fbe4409e490 2024-09-16T15:07:29.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe49268640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440a5910 con 0x7fbe440a1010 2024-09-16T15:07:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe42ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbe440a2cf0 0x7fbe440a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe437fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 0x7fbe44144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe42ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbe440a2cf0 0x7fbe440a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57426/0 (socket says 172.21.15.133:57426) 2024-09-16T15:07:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe42ffd640 1 -- 172.21.15.133:0/1939602292 learned_addr learned my addr 172.21.15.133:0/1939602292 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe3bfff640 1 --2- 172.21.15.133:0/1939602292 >> v2:172.21.15.191:3300/0 conn(0x7fbe4409e490 0x7fbe4406afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe437fe640 1 -- 172.21.15.133:0/1939602292 >> v2:172.21.15.191:3300/0 conn(0x7fbe4409e490 msgr2=0x7fbe4406afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe437fe640 1 --2- 172.21.15.133:0/1939602292 >> v2:172.21.15.191:3300/0 conn(0x7fbe4409e490 0x7fbe4406afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe437fe640 1 -- 172.21.15.133:0/1939602292 >> v2:172.21.15.133:3300/0 conn(0x7fbe440a2cf0 msgr2=0x7fbe440a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe437fe640 1 --2- 172.21.15.133:0/1939602292 >> v2:172.21.15.133:3300/0 conn(0x7fbe440a2cf0 0x7fbe440a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe437fe640 1 -- 172.21.15.133:0/1939602292 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe440a4cf0 con 0x7fbe440a1010 2024-09-16T15:07:29.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe42ffd640 1 --2- 172.21.15.133:0/1939602292 >> v2:172.21.15.133:3300/0 conn(0x7fbe440a2cf0 0x7fbe440a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe437fe640 1 --2- 172.21.15.133:0/1939602292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 0x7fbe44144a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe4409e420 tx=0x7fbe3c01c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc16a12733053e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe427fc640 1 -- 172.21.15.133:0/1939602292 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0a5870 con 0x7fbe440a1010 2024-09-16T15:07:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe427fc640 1 -- 172.21.15.133:0/1939602292 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c0a59d0 con 0x7fbe440a1010 2024-09-16T15:07:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.258+0000 7fbe427fc640 1 -- 172.21.15.133:0/1939602292 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0a5d60 con 0x7fbe440a1010 2024-09-16T15:07:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- 172.21.15.133:0/1939602292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 msgr2=0x7fbe44144a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 --2- 172.21.15.133:0/1939602292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 0x7fbe44144a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe4409e420 tx=0x7fbe3c01c7f0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- 172.21.15.133:0/1939602292 shutdown_connections 2024-09-16T15:07:29.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 --2- 172.21.15.133:0/1939602292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 0x7fbe44144a60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 --2- 172.21.15.133:0/1939602292 >> v2:172.21.15.191:3300/0 conn(0x7fbe4409e490 0x7fbe4406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 --2- 172.21.15.133:0/1939602292 >> v2:172.21.15.133:3300/0 conn(0x7fbe440a2cf0 0x7fbe440a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- 172.21.15.133:0/1939602292 >> 172.21.15.133:0/1939602292 conn(0x7fbe440a3fd0 msgr2=0x7fbe440a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- 172.21.15.133:0/1939602292 shutdown_connections 2024-09-16T15:07:29.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- 172.21.15.133:0/1939602292 wait complete. 2024-09-16T15:07:29.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 Processor -- start 2024-09-16T15:07:29.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- start start 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4409e490 0x7fbe44147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 0x7fbe4409d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe440a2cf0 0x7fbe441464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44146f10 con 0x7fbe4409e490 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441474e0 con 0x7fbe440a2cf0 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441df200 con 0x7fbe440a1010 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe42ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4409e490 0x7fbe44147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe42ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4409e490 0x7fbe44147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57434/0 (socket says 172.21.15.133:57434) 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe42ffd640 1 -- 172.21.15.133:0/2412301417 learned_addr learned my addr 172.21.15.133:0/2412301417 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe3bfff640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 0x7fbe4409d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe437fe640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe440a2cf0 0x7fbe441464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe42ffd640 1 -- 172.21.15.133:0/2412301417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 msgr2=0x7fbe4409d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe42ffd640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 0x7fbe4409d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe42ffd640 1 -- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe440a2cf0 msgr2=0x7fbe441464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe42ffd640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe440a2cf0 0x7fbe441464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe42ffd640 1 -- 172.21.15.133:0/2412301417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340094d0 con 0x7fbe4409e490 2024-09-16T15:07:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe3bfff640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 0x7fbe4409d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe437fe640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe440a2cf0 0x7fbe441464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe42ffd640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4409e490 0x7fbe44147c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400ea00 tx=0x7fbe3400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe3b7fe640 1 -- 172.21.15.133:0/2412301417 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3400ca60 con 0x7fbe4409e490 2024-09-16T15:07:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe3b7fe640 1 -- 172.21.15.133:0/2412301417 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3400cbc0 con 0x7fbe4409e490 2024-09-16T15:07:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe3b7fe640 1 -- 172.21.15.133:0/2412301417 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34013470 con 0x7fbe4409e490 2024-09-16T15:07:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- 172.21.15.133:0/2412301417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c009a70 con 0x7fbe4409e490 2024-09-16T15:07:29.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- 172.21.15.133:0/2412301417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4409d710 con 0x7fbe4409e490 2024-09-16T15:07:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.262+0000 7fbe49268640 1 -- 172.21.15.133:0/2412301417 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe44144a60 con 0x7fbe4409e490 2024-09-16T15:07:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.266+0000 7fbe3b7fe640 1 -- 172.21.15.133:0/2412301417 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fbe340135d0 con 0x7fbe4409e490 2024-09-16T15:07:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.266+0000 7fbe3b7fe640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.266+0000 7fbe3b7fe640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.266+0000 7fbe3b7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.266+0000 7fbe3b7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.266+0000 7fbe3b7fe640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fbe240978c0 0x7fbe24099d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.266+0000 7fbe3b7fe640 1 -- 172.21.15.133:0/2412301417 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7fbe340c00b0 con 0x7fbe4409e490 2024-09-16T15:07:29.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.266+0000 7fbe3bfff640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fbe240978c0 0x7fbe24099d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.266+0000 7fbe3bfff640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fbe240978c0 0x7fbe24099d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe44147380 tx=0x7fbe2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.270+0000 7fbe3b7fe640 1 -- 172.21.15.133:0/2412301417 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbe3408d720 con 0x7fbe4409e490 2024-09-16T15:07:29.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.414+0000 7fbe49268640 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2024-09-16T15:07:29.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.414+0000 7fbe49268640 1 -- 172.21.15.133:0/2412301417 --> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe4406ba00 con 0x7fbe240978c0 2024-09-16T15:07:29.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe3b7fe640 1 -- 172.21.15.133:0/2412301417 <== mgr.4102 v2:172.21.15.191:6824/3926506186 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271531 (secure 0 0 0) 0x7fbe4406ba00 con 0x7fbe240978c0 2024-09-16T15:07:29.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe3b7fe640 20 mgrc handle_command_reply tid 0 r 0 2024-09-16T15:07:29.441 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:29.442 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-16T15:07:29.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 10 mgrc shutdown 2024-09-16T15:07:29.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 -- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fbe240978c0 msgr2=0x7fbe24099d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fbe240978c0 0x7fbe24099d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbe44147380 tx=0x7fbe2c008040 comp rx=0 tx=0).stop 2024-09-16T15:07:29.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 -- 172.21.15.133:0/2412301417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4409e490 msgr2=0x7fbe44147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4409e490 0x7fbe44147c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400ea00 tx=0x7fbe3400eec0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 -- 172.21.15.133:0/2412301417 shutdown_connections 2024-09-16T15:07:29.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe440a2cf0 0x7fbe441464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe440a1010 0x7fbe4409d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fbe240978c0 0x7fbe24099d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 --2- 172.21.15.133:0/2412301417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4409e490 0x7fbe44147c50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 -- 172.21.15.133:0/2412301417 >> 172.21.15.133:0/2412301417 conn(0x7fbe440a3fd0 msgr2=0x7fbe440b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 -- 172.21.15.133:0/2412301417 shutdown_connections 2024-09-16T15:07:29.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.438+0000 7fbe49268640 1 -- 172.21.15.133:0/2412301417 wait complete. 2024-09-16T15:07:29.457 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-16T15:07:28.691398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592749,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110084,"kb_used_data":1984,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377377276,"statfs":{"total":386547056640,"available":386434330624,"internally_reserved":0,"allocated":2031616,"data_stored":1456554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6374,"internal_metadata":110618394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.640220"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774982+0000","last_change":"2024-09-16T15:07:22.073492+0000","last_active":"2024-09-16T15:07:22.774982+0000","last_peered":"2024-09-16T15:07:22.774982+0000","last_clean":"2024-09-16T15:07:22.774982+0000","last_became_active":"2024-09-16T15:07:22.073186+0000","last_became_peered":"2024-09-16T15:07:22.073186+0000","last_unstale":"2024-09-16T15:07:22.774982+0000","last_undegraded":"2024-09-16T15:07:22.774982+0000","last_fullsized":"2024-09-16T15:07:22.774982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774795+0000","last_change":"2024-09-16T15:07:22.089956+0000","last_active":"2024-09-16T15:07:22.774795+0000","last_peered":"2024-09-16T15:07:22.774795+0000","last_clean":"2024-09-16T15:07:22.774795+0000","last_became_active":"2024-09-16T15:07:22.089659+0000","last_became_peered":"2024-09-16T15:07:22.089659+0000","last_unstale":"2024-09-16T15:07:22.774795+0000","last_undegraded":"2024-09-16T15:07:22.774795+0000","last_fullsized":"2024-09-16T15:07:22.774795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:14.566411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060926+0000","last_change":"2024-09-16T15:07:22.092432+0000","last_active":"2024-09-16T15:07:23.060926+0000","last_peered":"2024-09-16T15:07:23.060926+0000","last_clean":"2024-09-16T15:07:23.060926+0000","last_became_active":"2024-09-16T15:07:22.092188+0000","last_became_peered":"2024-09-16T15:07:22.092188+0000","last_unstale":"2024-09-16T15:07:23.060926+0000","last_undegraded":"2024-09-16T15:07:23.060926+0000","last_fullsized":"2024-09-16T15:07:23.060926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:25.997418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773890+0000","last_change":"2024-09-16T15:07:22.095720+0000","last_active":"2024-09-16T15:07:22.773890+0000","last_peered":"2024-09-16T15:07:22.773890+0000","last_clean":"2024-09-16T15:07:22.773890+0000","last_became_active":"2024-09-16T15:07:22.093626+0000","last_became_peered":"2024-09-16T15:07:22.093626+0000","last_unstale":"2024-09-16T15:07:22.773890+0000","last_undegraded":"2024-09-16T15:07:22.773890+0000","last_fullsized":"2024-09-16T15:07:22.773890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:24.351732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060209+0000","last_change":"2024-09-16T15:07:22.077679+0000","last_active":"2024-09-16T15:07:23.060209+0000","last_peered":"2024-09-16T15:07:23.060209+0000","last_clean":"2024-09-16T15:07:23.060209+0000","last_became_active":"2024-09-16T15:07:22.077387+0000","last_became_peered":"2024-09-16T15:07:22.077387+0000","last_unstale":"2024-09-16T15:07:23.060209+0000","last_undegraded":"2024-09-16T15:07:23.060209+0000","last_fullsized":"2024-09-16T15:07:23.060209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:09.497997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059444+0000","last_change":"2024-09-16T15:07:22.077568+0000","last_active":"2024-09-16T15:07:23.059444+0000","last_peered":"2024-09-16T15:07:23.059444+0000","last_clean":"2024-09-16T15:07:23.059444+0000","last_became_active":"2024-09-16T15:07:22.076638+0000","last_became_peered":"2024-09-16T15:07:22.076638+0000","last_unstale":"2024-09-16T15:07:23.059444+0000","last_undegraded":"2024-09-16T15:07:23.059444+0000","last_fullsized":"2024-09-16T15:07:23.059444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:28.264456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.058994+0000","last_change":"2024-09-16T15:07:22.075898+0000","last_active":"2024-09-16T15:07:23.058994+0000","last_peered":"2024-09-16T15:07:23.058994+0000","last_clean":"2024-09-16T15:07:23.058994+0000","last_became_active":"2024-09-16T15:07:22.075448+0000","last_became_peered":"2024-09-16T15:07:22.075448+0000","last_unstale":"2024-09-16T15:07:23.058994+0000","last_undegraded":"2024-09-16T15:07:23.058994+0000","last_fullsized":"2024-09-16T15:07:23.058994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:03.093921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773305+0000","last_change":"2024-09-16T15:07:22.096662+0000","last_active":"2024-09-16T15:07:22.773305+0000","last_peered":"2024-09-16T15:07:22.773305+0000","last_clean":"2024-09-16T15:07:22.773305+0000","last_became_active":"2024-09-16T15:07:22.095671+0000","last_became_peered":"2024-09-16T15:07:22.095671+0000","last_unstale":"2024-09-16T15:07:22.773305+0000","last_undegraded":"2024-09-16T15:07:22.773305+0000","last_fullsized":"2024-09-16T15:07:22.773305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:47.108958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062345+0000","last_change":"2024-09-16T15:07:22.092184+0000","last_active":"2024-09-16T15:07:23.062345+0000","last_peered":"2024-09-16T15:07:23.062345+0000","last_clean":"2024-09-16T15:07:23.062345+0000","last_became_active":"2024-09-16T15:07:22.091896+0000","last_became_peered":"2024-09-16T15:07:22.091896+0000","last_unstale":"2024-09-16T15:07:23.062345+0000","last_undegraded":"2024-09-16T15:07:23.062345+0000","last_fullsized":"2024-09-16T15:07:23.062345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774013+0000","last_change":"2024-09-16T15:07:22.093487+0000","last_active":"2024-09-16T15:07:22.774013+0000","last_peered":"2024-09-16T15:07:22.774013+0000","last_clean":"2024-09-16T15:07:22.774013+0000","last_became_active":"2024-09-16T15:07:22.091134+0000","last_became_peered":"2024-09-16T15:07:22.091134+0000","last_unstale":"2024-09-16T15:07:22.774013+0000","last_undegraded":"2024-09-16T15:07:22.774013+0000","last_fullsized":"2024-09-16T15:07:22.774013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:55.259949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773989+0000","last_change":"2024-09-16T15:07:22.091123+0000","last_active":"2024-09-16T15:07:22.773989+0000","last_peered":"2024-09-16T15:07:22.773989+0000","last_clean":"2024-09-16T15:07:22.773989+0000","last_became_active":"2024-09-16T15:07:22.090822+0000","last_became_peered":"2024-09-16T15:07:22.090822+0000","last_unstale":"2024-09-16T15:07:22.773989+0000","last_undegraded":"2024-09-16T15:07:22.773989+0000","last_fullsized":"2024-09-16T15:07:22.773989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774722+0000","last_change":"2024-09-16T15:07:22.075137+0000","last_active":"2024-09-16T15:07:22.774722+0000","last_peered":"2024-09-16T15:07:22.774722+0000","last_clean":"2024-09-16T15:07:22.774722+0000","last_became_active":"2024-09-16T15:07:22.074779+0000","last_became_peered":"2024-09-16T15:07:22.074779+0000","last_unstale":"2024-09-16T15:07:22.774722+0000","last_undegraded":"2024-09-16T15:07:22.774722+0000","last_fullsized":"2024-09-16T15:07:22.774722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:34.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061606+0000","last_change":"2024-09-16T15:07:22.069681+0000","last_active":"2024-09-16T15:07:23.061606+0000","last_peered":"2024-09-16T15:07:23.061606+0000","last_clean":"2024-09-16T15:07:23.061606+0000","last_became_active":"2024-09-16T15:07:22.069331+0000","last_became_peered":"2024-09-16T15:07:22.069331+0000","last_unstale":"2024-09-16T15:07:23.061606+0000","last_undegraded":"2024-09-16T15:07:23.061606+0000","last_fullsized":"2024-09-16T15:07:23.061606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:27.703305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773541+0000","last_change":"2024-09-16T15:07:22.093463+0000","last_active":"2024-09-16T15:07:22.773541+0000","last_peered":"2024-09-16T15:07:22.773541+0000","last_clean":"2024-09-16T15:07:22.773541+0000","last_became_active":"2024-09-16T15:07:22.091154+0000","last_became_peered":"2024-09-16T15:07:22.091154+0000","last_unstale":"2024-09-16T15:07:22.773541+0000","last_undegraded":"2024-09-16T15:07:22.773541+0000","last_fullsized":"2024-09-16T15:07:22.773541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:24:09.086333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773431+0000","last_change":"2024-09-16T15:07:22.087589+0000","last_active":"2024-09-16T15:07:22.773431+0000","last_peered":"2024-09-16T15:07:22.773431+0000","last_clean":"2024-09-16T15:07:22.773431+0000","last_became_active":"2024-09-16T15:07:22.087167+0000","last_became_peered":"2024-09-16T15:07:22.087167+0000","last_unstale":"2024-09-16T15:07:22.773431+0000","last_undegraded":"2024-09-16T15:07:22.773431+0000","last_fullsized":"2024-09-16T15:07:22.773431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:26.349524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.058848+0000","last_change":"2024-09-16T15:07:22.076065+0000","last_active":"2024-09-16T15:07:23.058848+0000","last_peered":"2024-09-16T15:07:23.058848+0000","last_clean":"2024-09-16T15:07:23.058848+0000","last_became_active":"2024-09-16T15:07:22.075724+0000","last_became_peered":"2024-09-16T15:07:22.075724+0000","last_unstale":"2024-09-16T15:07:23.058848+0000","last_undegraded":"2024-09-16T15:07:23.058848+0000","last_fullsized":"2024-09-16T15:07:23.058848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:20:30.555559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774060+0000","last_change":"2024-09-16T15:07:22.091007+0000","last_active":"2024-09-16T15:07:22.774060+0000","last_peered":"2024-09-16T15:07:22.774060+0000","last_clean":"2024-09-16T15:07:22.774060+0000","last_became_active":"2024-09-16T15:07:22.089216+0000","last_became_peered":"2024-09-16T15:07:22.089216+0000","last_unstale":"2024-09-16T15:07:22.774060+0000","last_undegraded":"2024-09-16T15:07:22.774060+0000","last_fullsized":"2024-09-16T15:07:22.774060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:43.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061475+0000","last_change":"2024-09-16T15:07:22.077822+0000","last_active":"2024-09-16T15:07:23.061475+0000","last_peered":"2024-09-16T15:07:23.061475+0000","last_clean":"2024-09-16T15:07:23.061475+0000","last_became_active":"2024-09-16T15:07:22.076545+0000","last_became_peered":"2024-09-16T15:07:22.076545+0000","last_unstale":"2024-09-16T15:07:23.061475+0000","last_undegraded":"2024-09-16T15:07:23.061475+0000","last_fullsized":"2024-09-16T15:07:23.061475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:03.892835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061811+0000","last_change":"2024-09-16T15:07:22.074503+0000","last_active":"2024-09-16T15:07:23.061811+0000","last_peered":"2024-09-16T15:07:23.061811+0000","last_clean":"2024-09-16T15:07:23.061811+0000","last_became_active":"2024-09-16T15:07:22.074240+0000","last_became_peered":"2024-09-16T15:07:22.074240+0000","last_unstale":"2024-09-16T15:07:23.061811+0000","last_undegraded":"2024-09-16T15:07:23.061811+0000","last_fullsized":"2024-09-16T15:07:23.061811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:45.906789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773917+0000","last_change":"2024-09-16T15:07:22.090412+0000","last_active":"2024-09-16T15:07:22.773917+0000","last_peered":"2024-09-16T15:07:22.773917+0000","last_clean":"2024-09-16T15:07:22.773917+0000","last_became_active":"2024-09-16T15:07:22.089178+0000","last_became_peered":"2024-09-16T15:07:22.089178+0000","last_unstale":"2024-09-16T15:07:22.773917+0000","last_undegraded":"2024-09-16T15:07:22.773917+0000","last_fullsized":"2024-09-16T15:07:22.773917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:20.559503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773814+0000","last_change":"2024-09-16T15:07:22.085622+0000","last_active":"2024-09-16T15:07:22.773814+0000","last_peered":"2024-09-16T15:07:22.773814+0000","last_clean":"2024-09-16T15:07:22.773814+0000","last_became_active":"2024-09-16T15:07:22.085321+0000","last_became_peered":"2024-09-16T15:07:22.085321+0000","last_unstale":"2024-09-16T15:07:22.773814+0000","last_undegraded":"2024-09-16T15:07:22.773814+0000","last_fullsized":"2024-09-16T15:07:22.773814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:25.939312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773588+0000","last_change":"2024-09-16T15:07:22.088691+0000","last_active":"2024-09-16T15:07:22.773588+0000","last_peered":"2024-09-16T15:07:22.773588+0000","last_clean":"2024-09-16T15:07:22.773588+0000","last_became_active":"2024-09-16T15:07:22.085505+0000","last_became_peered":"2024-09-16T15:07:22.085505+0000","last_unstale":"2024-09-16T15:07:22.773588+0000","last_undegraded":"2024-09-16T15:07:22.773588+0000","last_fullsized":"2024-09-16T15:07:22.773588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:20.387713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059028+0000","last_change":"2024-09-16T15:07:22.093287+0000","last_active":"2024-09-16T15:07:23.059028+0000","last_peered":"2024-09-16T15:07:23.059028+0000","last_clean":"2024-09-16T15:07:23.059028+0000","last_became_active":"2024-09-16T15:07:22.092983+0000","last_became_peered":"2024-09-16T15:07:22.092983+0000","last_unstale":"2024-09-16T15:07:23.059028+0000","last_undegraded":"2024-09-16T15:07:23.059028+0000","last_fullsized":"2024-09-16T15:07:23.059028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:32.532671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773341+0000","last_change":"2024-09-16T15:07:22.096232+0000","last_active":"2024-09-16T15:07:22.773341+0000","last_peered":"2024-09-16T15:07:22.773341+0000","last_clean":"2024-09-16T15:07:22.773341+0000","last_became_active":"2024-09-16T15:07:22.091234+0000","last_became_peered":"2024-09-16T15:07:22.091234+0000","last_unstale":"2024-09-16T15:07:22.773341+0000","last_undegraded":"2024-09-16T15:07:22.773341+0000","last_fullsized":"2024-09-16T15:07:22.773341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:23.826942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774094+0000","last_change":"2024-09-16T15:07:22.088597+0000","last_active":"2024-09-16T15:07:22.774094+0000","last_peered":"2024-09-16T15:07:22.774094+0000","last_clean":"2024-09-16T15:07:22.774094+0000","last_became_active":"2024-09-16T15:07:22.086951+0000","last_became_peered":"2024-09-16T15:07:22.086951+0000","last_unstale":"2024-09-16T15:07:22.774094+0000","last_undegraded":"2024-09-16T15:07:22.774094+0000","last_fullsized":"2024-09-16T15:07:22.774094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:57.389973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062294+0000","last_change":"2024-09-16T15:07:22.067847+0000","last_active":"2024-09-16T15:07:23.062294+0000","last_peered":"2024-09-16T15:07:23.062294+0000","last_clean":"2024-09-16T15:07:23.062294+0000","last_became_active":"2024-09-16T15:07:22.067337+0000","last_became_peered":"2024-09-16T15:07:22.067337+0000","last_unstale":"2024-09-16T15:07:23.062294+0000","last_undegraded":"2024-09-16T15:07:23.062294+0000","last_fullsized":"2024-09-16T15:07:23.062294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:23.826942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061869+0000","last_change":"2024-09-16T15:07:22.079269+0000","last_active":"2024-09-16T15:07:23.061869+0000","last_peered":"2024-09-16T15:07:23.061869+0000","last_clean":"2024-09-16T15:07:23.061869+0000","last_became_active":"2024-09-16T15:07:22.078377+0000","last_became_peered":"2024-09-16T15:07:22.078377+0000","last_unstale":"2024-09-16T15:07:23.061869+0000","last_undegraded":"2024-09-16T15:07:23.061869+0000","last_fullsized":"2024-09-16T15:07:23.061869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:51.357090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061889+0000","last_change":"2024-09-16T15:07:22.067944+0000","last_active":"2024-09-16T15:07:23.061889+0000","last_peered":"2024-09-16T15:07:23.061889+0000","last_clean":"2024-09-16T15:07:23.061889+0000","last_became_active":"2024-09-16T15:07:22.067670+0000","last_became_peered":"2024-09-16T15:07:22.067670+0000","last_unstale":"2024-09-16T15:07:23.061889+0000","last_undegraded":"2024-09-16T15:07:23.061889+0000","last_fullsized":"2024-09-16T15:07:23.061889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:10.707281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061839+0000","last_change":"2024-09-16T15:07:22.074211+0000","last_active":"2024-09-16T15:07:23.061839+0000","last_peered":"2024-09-16T15:07:23.061839+0000","last_clean":"2024-09-16T15:07:23.061839+0000","last_became_active":"2024-09-16T15:07:22.073954+0000","last_became_peered":"2024-09-16T15:07:22.073954+0000","last_unstale":"2024-09-16T15:07:23.061839+0000","last_undegraded":"2024-09-16T15:07:23.061839+0000","last_fullsized":"2024-09-16T15:07:23.061839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:51.922251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773716+0000","last_change":"2024-09-16T15:07:22.088989+0000","last_active":"2024-09-16T15:07:22.773716+0000","last_peered":"2024-09-16T15:07:22.773716+0000","last_clean":"2024-09-16T15:07:22.773716+0000","last_became_active":"2024-09-16T15:07:22.085895+0000","last_became_peered":"2024-09-16T15:07:22.085895+0000","last_unstale":"2024-09-16T15:07:22.773716+0000","last_undegraded":"2024-09-16T15:07:22.773716+0000","last_fullsized":"2024-09-16T15:07:22.773716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:58.765327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060913+0000","last_change":"2024-09-16T15:07:22.092307+0000","last_active":"2024-09-16T15:07:23.060913+0000","last_peered":"2024-09-16T15:07:23.060913+0000","last_clean":"2024-09-16T15:07:23.060913+0000","last_became_active":"2024-09-16T15:07:22.091941+0000","last_became_peered":"2024-09-16T15:07:22.091941+0000","last_unstale":"2024-09-16T15:07:23.060913+0000","last_undegraded":"2024-09-16T15:07:23.060913+0000","last_fullsized":"2024-09-16T15:07:23.060913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:14.566411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773367+0000","last_change":"2024-09-16T15:07:22.096217+0000","last_active":"2024-09-16T15:07:22.773367+0000","last_peered":"2024-09-16T15:07:22.773367+0000","last_clean":"2024-09-16T15:07:22.773367+0000","last_became_active":"2024-09-16T15:07:22.092983+0000","last_became_peered":"2024-09-16T15:07:22.092983+0000","last_unstale":"2024-09-16T15:07:22.773367+0000","last_undegraded":"2024-09-16T15:07:22.773367+0000","last_fullsized":"2024-09-16T15:07:22.773367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:04.412982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060607+0000","last_change":"2024-09-16T15:07:22.079554+0000","last_active":"2024-09-16T15:07:23.060607+0000","last_peered":"2024-09-16T15:07:23.060607+0000","last_clean":"2024-09-16T15:07:23.060607+0000","last_became_active":"2024-09-16T15:07:22.078495+0000","last_became_peered":"2024-09-16T15:07:22.078495+0000","last_unstale":"2024-09-16T15:07:23.060607+0000","last_undegraded":"2024-09-16T15:07:23.060607+0000","last_fullsized":"2024-09-16T15:07:23.060607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:25.939312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060889+0000","last_change":"2024-09-16T15:07:21.742735+0000","last_active":"2024-09-16T15:07:23.060889+0000","last_peered":"2024-09-16T15:07:23.060889+0000","last_clean":"2024-09-16T15:07:23.060889+0000","last_became_active":"2024-09-16T15:07:21.742323+0000","last_became_peered":"2024-09-16T15:07:21.742323+0000","last_unstale":"2024-09-16T15:07:23.060889+0000","last_undegraded":"2024-09-16T15:07:23.060889+0000","last_fullsized":"2024-09-16T15:07:23.060889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:09.704280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061565+0000","last_change":"2024-09-16T15:07:22.079328+0000","last_active":"2024-09-16T15:07:23.061565+0000","last_peered":"2024-09-16T15:07:23.061565+0000","last_clean":"2024-09-16T15:07:23.061565+0000","last_became_active":"2024-09-16T15:07:22.078344+0000","last_became_peered":"2024-09-16T15:07:22.078344+0000","last_unstale":"2024-09-16T15:07:23.061565+0000","last_undegraded":"2024-09-16T15:07:23.061565+0000","last_fullsized":"2024-09-16T15:07:23.061565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:22:23.839319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774595+0000","last_change":"2024-09-16T15:07:21.737802+0000","last_active":"2024-09-16T15:07:22.774595+0000","last_peered":"2024-09-16T15:07:22.774595+0000","last_clean":"2024-09-16T15:07:22.774595+0000","last_became_active":"2024-09-16T15:07:21.737174+0000","last_became_peered":"2024-09-16T15:07:21.737174+0000","last_unstale":"2024-09-16T15:07:22.774595+0000","last_undegraded":"2024-09-16T15:07:22.774595+0000","last_fullsized":"2024-09-16T15:07:22.774595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061591+0000","last_change":"2024-09-16T15:07:22.079075+0000","last_active":"2024-09-16T15:07:23.061591+0000","last_peered":"2024-09-16T15:07:23.061591+0000","last_clean":"2024-09-16T15:07:23.061591+0000","last_became_active":"2024-09-16T15:07:22.077520+0000","last_became_peered":"2024-09-16T15:07:22.077520+0000","last_unstale":"2024-09-16T15:07:23.061591+0000","last_undegraded":"2024-09-16T15:07:23.061591+0000","last_fullsized":"2024-09-16T15:07:23.061591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:29.976832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060410+0000","last_change":"2024-09-16T15:07:21.739614+0000","last_active":"2024-09-16T15:07:23.060410+0000","last_peered":"2024-09-16T15:07:23.060410+0000","last_clean":"2024-09-16T15:07:23.060410+0000","last_became_active":"2024-09-16T15:07:21.739226+0000","last_became_peered":"2024-09-16T15:07:21.739226+0000","last_unstale":"2024-09-16T15:07:23.060410+0000","last_undegraded":"2024-09-16T15:07:23.060410+0000","last_fullsized":"2024-09-16T15:07:23.060410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:13.563410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062264+0000","last_change":"2024-09-16T15:07:22.087272+0000","last_active":"2024-09-16T15:07:23.062264+0000","last_peered":"2024-09-16T15:07:23.062264+0000","last_clean":"2024-09-16T15:07:23.062264+0000","last_became_active":"2024-09-16T15:07:22.086953+0000","last_became_peered":"2024-09-16T15:07:22.086953+0000","last_unstale":"2024-09-16T15:07:23.062264+0000","last_undegraded":"2024-09-16T15:07:23.062264+0000","last_fullsized":"2024-09-16T15:07:23.062264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:34.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773390+0000","last_change":"2024-09-16T15:07:21.733899+0000","last_active":"2024-09-16T15:07:22.773390+0000","last_peered":"2024-09-16T15:07:22.773390+0000","last_clean":"2024-09-16T15:07:22.773390+0000","last_became_active":"2024-09-16T15:07:21.730134+0000","last_became_peered":"2024-09-16T15:07:21.730134+0000","last_unstale":"2024-09-16T15:07:22.773390+0000","last_undegraded":"2024-09-16T15:07:22.773390+0000","last_fullsized":"2024-09-16T15:07:22.773390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.775026+0000","last_change":"2024-09-16T15:07:21.720627+0000","last_active":"2024-09-16T15:07:22.775026+0000","last_peered":"2024-09-16T15:07:22.775026+0000","last_clean":"2024-09-16T15:07:22.775026+0000","last_became_active":"2024-09-16T15:07:21.719903+0000","last_became_peered":"2024-09-16T15:07:21.719903+0000","last_unstale":"2024-09-16T15:07:22.775026+0000","last_undegraded":"2024-09-16T15:07:22.775026+0000","last_fullsized":"2024-09-16T15:07:22.775026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059146+0000","last_change":"2024-09-16T15:07:22.100423+0000","last_active":"2024-09-16T15:07:23.059146+0000","last_peered":"2024-09-16T15:07:23.059146+0000","last_clean":"2024-09-16T15:07:23.059146+0000","last_became_active":"2024-09-16T15:07:22.099384+0000","last_became_peered":"2024-09-16T15:07:22.099384+0000","last_unstale":"2024-09-16T15:07:23.059146+0000","last_undegraded":"2024-09-16T15:07:23.059146+0000","last_fullsized":"2024-09-16T15:07:23.059146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:15.595055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774917+0000","last_change":"2024-09-16T15:07:21.720105+0000","last_active":"2024-09-16T15:07:22.774917+0000","last_peered":"2024-09-16T15:07:22.774917+0000","last_clean":"2024-09-16T15:07:22.774917+0000","last_became_active":"2024-09-16T15:07:21.719254+0000","last_became_peered":"2024-09-16T15:07:21.719254+0000","last_unstale":"2024-09-16T15:07:22.774917+0000","last_undegraded":"2024-09-16T15:07:22.774917+0000","last_fullsized":"2024-09-16T15:07:22.774917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773453+0000","last_change":"2024-09-16T15:07:22.087123+0000","last_active":"2024-09-16T15:07:22.773453+0000","last_peered":"2024-09-16T15:07:22.773453+0000","last_clean":"2024-09-16T15:07:22.773453+0000","last_became_active":"2024-09-16T15:07:22.085858+0000","last_became_peered":"2024-09-16T15:07:22.085858+0000","last_unstale":"2024-09-16T15:07:22.773453+0000","last_undegraded":"2024-09-16T15:07:22.773453+0000","last_fullsized":"2024-09-16T15:07:22.773453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:43.560388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061868+0000","last_change":"2024-09-16T15:07:21.727975+0000","last_active":"2024-09-16T15:07:23.061868+0000","last_peered":"2024-09-16T15:07:23.061868+0000","last_clean":"2024-09-16T15:07:23.061868+0000","last_became_active":"2024-09-16T15:07:21.726999+0000","last_became_peered":"2024-09-16T15:07:21.726999+0000","last_unstale":"2024-09-16T15:07:23.061868+0000","last_undegraded":"2024-09-16T15:07:23.061868+0000","last_fullsized":"2024-09-16T15:07:23.061868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:26.627634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774542+0000","last_change":"2024-09-16T15:07:22.064842+0000","last_active":"2024-09-16T15:07:22.774542+0000","last_peered":"2024-09-16T15:07:22.774542+0000","last_clean":"2024-09-16T15:07:22.774542+0000","last_became_active":"2024-09-16T15:07:22.064502+0000","last_became_peered":"2024-09-16T15:07:22.064502+0000","last_unstale":"2024-09-16T15:07:22.774542+0000","last_undegraded":"2024-09-16T15:07:22.774542+0000","last_fullsized":"2024-09-16T15:07:22.774542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:51.922251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061051+0000","last_change":"2024-09-16T15:07:21.725945+0000","last_active":"2024-09-16T15:07:23.061051+0000","last_peered":"2024-09-16T15:07:23.061051+0000","last_clean":"2024-09-16T15:07:23.061051+0000","last_became_active":"2024-09-16T15:07:21.724839+0000","last_became_peered":"2024-09-16T15:07:21.724839+0000","last_unstale":"2024-09-16T15:07:23.061051+0000","last_undegraded":"2024-09-16T15:07:23.061051+0000","last_fullsized":"2024-09-16T15:07:23.061051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:47.932258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774660+0000","last_change":"2024-09-16T15:07:22.058944+0000","last_active":"2024-09-16T15:07:22.774660+0000","last_peered":"2024-09-16T15:07:22.774660+0000","last_clean":"2024-09-16T15:07:22.774660+0000","last_became_active":"2024-09-16T15:07:22.058700+0000","last_became_peered":"2024-09-16T15:07:22.058700+0000","last_unstale":"2024-09-16T15:07:22.774660+0000","last_undegraded":"2024-09-16T15:07:22.774660+0000","last_fullsized":"2024-09-16T15:07:22.774660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:10.707281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061840+0000","last_change":"2024-09-16T15:07:21.725144+0000","last_active":"2024-09-16T15:07:23.061840+0000","last_peered":"2024-09-16T15:07:23.061840+0000","last_clean":"2024-09-16T15:07:23.061840+0000","last_became_active":"2024-09-16T15:07:21.724855+0000","last_became_peered":"2024-09-16T15:07:21.724855+0000","last_unstale":"2024-09-16T15:07:23.061840+0000","last_undegraded":"2024-09-16T15:07:23.061840+0000","last_fullsized":"2024-09-16T15:07:23.061840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773940+0000","last_change":"2024-09-16T15:07:22.087348+0000","last_active":"2024-09-16T15:07:22.773940+0000","last_peered":"2024-09-16T15:07:22.773940+0000","last_clean":"2024-09-16T15:07:22.773940+0000","last_became_active":"2024-09-16T15:07:22.086906+0000","last_became_peered":"2024-09-16T15:07:22.086906+0000","last_unstale":"2024-09-16T15:07:22.773940+0000","last_undegraded":"2024-09-16T15:07:22.773940+0000","last_fullsized":"2024-09-16T15:07:22.773940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:53.121258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059464+0000","last_change":"2024-09-16T15:07:21.727272+0000","last_active":"2024-09-16T15:07:23.059464+0000","last_peered":"2024-09-16T15:07:23.059464+0000","last_clean":"2024-09-16T15:07:23.059464+0000","last_became_active":"2024-09-16T15:07:21.724280+0000","last_became_peered":"2024-09-16T15:07:21.724280+0000","last_unstale":"2024-09-16T15:07:23.059464+0000","last_undegraded":"2024-09-16T15:07:23.059464+0000","last_fullsized":"2024-09-16T15:07:23.059464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:50.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061388+0000","last_change":"2024-09-16T15:07:22.094628+0000","last_active":"2024-09-16T15:07:23.061388+0000","last_peered":"2024-09-16T15:07:23.061388+0000","last_clean":"2024-09-16T15:07:23.061388+0000","last_became_active":"2024-09-16T15:07:22.094356+0000","last_became_peered":"2024-09-16T15:07:22.094356+0000","last_unstale":"2024-09-16T15:07:23.061388+0000","last_undegraded":"2024-09-16T15:07:23.061388+0000","last_fullsized":"2024-09-16T15:07:23.061388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:37.486620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060474+0000","last_change":"2024-09-16T15:07:21.742014+0000","last_active":"2024-09-16T15:07:23.060474+0000","last_peered":"2024-09-16T15:07:23.060474+0000","last_clean":"2024-09-16T15:07:23.060474+0000","last_became_active":"2024-09-16T15:07:21.741361+0000","last_became_peered":"2024-09-16T15:07:21.741361+0000","last_unstale":"2024-09-16T15:07:23.060474+0000","last_undegraded":"2024-09-16T15:07:23.060474+0000","last_fullsized":"2024-09-16T15:07:23.060474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:05.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774833+0000","last_change":"2024-09-16T15:07:22.072156+0000","last_active":"2024-09-16T15:07:22.774833+0000","last_peered":"2024-09-16T15:07:22.774833+0000","last_clean":"2024-09-16T15:07:22.774833+0000","last_became_active":"2024-09-16T15:07:22.071767+0000","last_became_peered":"2024-09-16T15:07:22.071767+0000","last_unstale":"2024-09-16T15:07:22.774833+0000","last_undegraded":"2024-09-16T15:07:22.774833+0000","last_fullsized":"2024-09-16T15:07:22.774833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:55.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059660+0000","last_change":"2024-09-16T15:07:21.726805+0000","last_active":"2024-09-16T15:07:23.059660+0000","last_peered":"2024-09-16T15:07:23.059660+0000","last_clean":"2024-09-16T15:07:23.059660+0000","last_became_active":"2024-09-16T15:07:21.726154+0000","last_became_peered":"2024-09-16T15:07:21.726154+0000","last_unstale":"2024-09-16T15:07:23.059660+0000","last_undegraded":"2024-09-16T15:07:23.059660+0000","last_fullsized":"2024-09-16T15:07:23.059660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:21.077487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774116+0000","last_change":"2024-09-16T15:07:22.088691+0000","last_active":"2024-09-16T15:07:22.774116+0000","last_peered":"2024-09-16T15:07:22.774116+0000","last_clean":"2024-09-16T15:07:22.774116+0000","last_became_active":"2024-09-16T15:07:22.087008+0000","last_became_peered":"2024-09-16T15:07:22.087008+0000","last_unstale":"2024-09-16T15:07:22.774116+0000","last_undegraded":"2024-09-16T15:07:22.774116+0000","last_fullsized":"2024-09-16T15:07:22.774116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:07.641636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":167,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.091073+0000","last_change":"2024-09-16T15:07:11.063383+0000","last_active":"2024-09-16T15:07:22.091073+0000","last_peered":"2024-09-16T15:07:22.091073+0000","last_clean":"2024-09-16T15:07:22.091073+0000","last_became_active":"2024-09-16T15:07:11.062607+0000","last_became_peered":"2024-09-16T15:07:11.062607+0000","last_unstale":"2024-09-16T15:07:22.091073+0000","last_undegraded":"2024-09-16T15:07:22.091073+0000","last_fullsized":"2024-09-16T15:07:22.091073+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:09.570910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:09.570910+0000","last_clean_scrub_stamp":"2024-09-16T15:07:09.570910+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:23.466827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.237221+0000","last_change":"2024-09-16T15:07:17.669610+0000","last_active":"2024-09-16T15:07:27.237221+0000","last_peered":"2024-09-16T15:07:27.237221+0000","last_clean":"2024-09-16T15:07:27.237221+0000","last_became_active":"2024-09-16T15:07:15.657618+0000","last_became_peered":"2024-09-16T15:07:15.657618+0000","last_unstale":"2024-09-16T15:07:27.237221+0000","last_undegraded":"2024-09-16T15:07:27.237221+0000","last_fullsized":"2024-09-16T15:07:27.237221+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:28:19.381043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085582900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059886+0000","last_change":"2024-09-16T15:07:21.741970+0000","last_active":"2024-09-16T15:07:23.059886+0000","last_peered":"2024-09-16T15:07:23.059886+0000","last_clean":"2024-09-16T15:07:23.059886+0000","last_became_active":"2024-09-16T15:07:21.741305+0000","last_became_peered":"2024-09-16T15:07:21.741305+0000","last_unstale":"2024-09-16T15:07:23.059886+0000","last_undegraded":"2024-09-16T15:07:23.059886+0000","last_fullsized":"2024-09-16T15:07:23.059886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:46.566147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061679+0000","last_change":"2024-09-16T15:07:22.095031+0000","last_active":"2024-09-16T15:07:23.061679+0000","last_peered":"2024-09-16T15:07:23.061679+0000","last_clean":"2024-09-16T15:07:23.061679+0000","last_became_active":"2024-09-16T15:07:22.094793+0000","last_became_peered":"2024-09-16T15:07:22.094793+0000","last_unstale":"2024-09-16T15:07:23.061679+0000","last_undegraded":"2024-09-16T15:07:23.061679+0000","last_fullsized":"2024-09-16T15:07:23.061679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:38:36.377299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.235522+0000","last_change":"2024-09-16T15:07:17.669433+0000","last_active":"2024-09-16T15:07:27.235522+0000","last_peered":"2024-09-16T15:07:27.235522+0000","last_clean":"2024-09-16T15:07:27.235522+0000","last_became_active":"2024-09-16T15:07:15.656237+0000","last_became_peered":"2024-09-16T15:07:15.656237+0000","last_unstale":"2024-09-16T15:07:27.235522+0000","last_undegraded":"2024-09-16T15:07:27.235522+0000","last_fullsized":"2024-09-16T15:07:27.235522+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:49.124474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068265100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061086+0000","last_change":"2024-09-16T15:07:21.713506+0000","last_active":"2024-09-16T15:07:23.061086+0000","last_peered":"2024-09-16T15:07:23.061086+0000","last_clean":"2024-09-16T15:07:23.061086+0000","last_became_active":"2024-09-16T15:07:21.713226+0000","last_became_peered":"2024-09-16T15:07:21.713226+0000","last_unstale":"2024-09-16T15:07:23.061086+0000","last_undegraded":"2024-09-16T15:07:23.061086+0000","last_fullsized":"2024-09-16T15:07:23.061086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061755+0000","last_change":"2024-09-16T15:07:22.079061+0000","last_active":"2024-09-16T15:07:23.061755+0000","last_peered":"2024-09-16T15:07:23.061755+0000","last_clean":"2024-09-16T15:07:23.061755+0000","last_became_active":"2024-09-16T15:07:22.076697+0000","last_became_peered":"2024-09-16T15:07:22.076697+0000","last_unstale":"2024-09-16T15:07:23.061755+0000","last_undegraded":"2024-09-16T15:07:23.061755+0000","last_fullsized":"2024-09-16T15:07:23.061755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:58.996879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.054623+0000","last_change":"2024-09-16T15:07:18.173036+0000","last_active":"2024-09-16T15:07:22.054623+0000","last_peered":"2024-09-16T15:07:22.054623+0000","last_clean":"2024-09-16T15:07:22.054623+0000","last_became_active":"2024-09-16T15:07:15.656238+0000","last_became_peered":"2024-09-16T15:07:15.656238+0000","last_unstale":"2024-09-16T15:07:22.054623+0000","last_undegraded":"2024-09-16T15:07:22.054623+0000","last_fullsized":"2024-09-16T15:07:22.054623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:15:02.977857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080979399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060974+0000","last_change":"2024-09-16T15:07:21.722652+0000","last_active":"2024-09-16T15:07:23.060974+0000","last_peered":"2024-09-16T15:07:23.060974+0000","last_clean":"2024-09-16T15:07:23.060974+0000","last_became_active":"2024-09-16T15:07:21.722308+0000","last_became_peered":"2024-09-16T15:07:21.722308+0000","last_unstale":"2024-09-16T15:07:23.060974+0000","last_undegraded":"2024-09-16T15:07:23.060974+0000","last_fullsized":"2024-09-16T15:07:23.060974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062351+0000","last_change":"2024-09-16T15:07:22.087268+0000","last_active":"2024-09-16T15:07:23.062351+0000","last_peered":"2024-09-16T15:07:23.062351+0000","last_clean":"2024-09-16T15:07:23.062351+0000","last_became_active":"2024-09-16T15:07:22.086932+0000","last_became_peered":"2024-09-16T15:07:22.086932+0000","last_unstale":"2024-09-16T15:07:23.062351+0000","last_undegraded":"2024-09-16T15:07:23.062351+0000","last_fullsized":"2024-09-16T15:07:23.062351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:55.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.236329+0000","last_change":"2024-09-16T15:07:17.671010+0000","last_active":"2024-09-16T15:07:27.236329+0000","last_peered":"2024-09-16T15:07:27.236329+0000","last_clean":"2024-09-16T15:07:27.236329+0000","last_became_active":"2024-09-16T15:07:15.658625+0000","last_became_peered":"2024-09-16T15:07:15.658625+0000","last_unstale":"2024-09-16T15:07:27.236329+0000","last_undegraded":"2024-09-16T15:07:27.236329+0000","last_fullsized":"2024-09-16T15:07:27.236329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:18.695305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00224435,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774699+0000","last_change":"2024-09-16T15:07:21.712013+0000","last_active":"2024-09-16T15:07:22.774699+0000","last_peered":"2024-09-16T15:07:22.774699+0000","last_clean":"2024-09-16T15:07:22.774699+0000","last_became_active":"2024-09-16T15:07:21.710551+0000","last_became_peered":"2024-09-16T15:07:21.710551+0000","last_unstale":"2024-09-16T15:07:22.774699+0000","last_undegraded":"2024-09-16T15:07:22.774699+0000","last_fullsized":"2024-09-16T15:07:22.774699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061756+0000","last_change":"2024-09-16T15:07:22.067593+0000","last_active":"2024-09-16T15:07:23.061756+0000","last_peered":"2024-09-16T15:07:23.061756+0000","last_clean":"2024-09-16T15:07:23.061756+0000","last_became_active":"2024-09-16T15:07:22.067254+0000","last_became_peered":"2024-09-16T15:07:22.067254+0000","last_unstale":"2024-09-16T15:07:23.061756+0000","last_undegraded":"2024-09-16T15:07:23.061756+0000","last_fullsized":"2024-09-16T15:07:23.061756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:43.560388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774774+0000","last_change":"2024-09-16T15:07:21.711346+0000","last_active":"2024-09-16T15:07:22.774774+0000","last_peered":"2024-09-16T15:07:22.774774+0000","last_clean":"2024-09-16T15:07:22.774774+0000","last_became_active":"2024-09-16T15:07:21.710693+0000","last_became_peered":"2024-09-16T15:07:21.710693+0000","last_unstale":"2024-09-16T15:07:22.774774+0000","last_undegraded":"2024-09-16T15:07:22.774774+0000","last_fullsized":"2024-09-16T15:07:22.774774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:50.814161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.234416+0000","last_change":"2024-09-16T15:07:17.664832+0000","last_active":"2024-09-16T15:07:27.234416+0000","last_peered":"2024-09-16T15:07:27.234416+0000","last_clean":"2024-09-16T15:07:27.234416+0000","last_became_active":"2024-09-16T15:07:15.654955+0000","last_became_peered":"2024-09-16T15:07:15.654955+0000","last_unstale":"2024-09-16T15:07:27.234416+0000","last_undegraded":"2024-09-16T15:07:27.234416+0000","last_fullsized":"2024-09-16T15:07:27.234416+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:48:38.399733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085315100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773630+0000","last_change":"2024-09-16T15:07:22.091087+0000","last_active":"2024-09-16T15:07:22.773630+0000","last_peered":"2024-09-16T15:07:22.773630+0000","last_clean":"2024-09-16T15:07:22.773630+0000","last_became_active":"2024-09-16T15:07:22.088585+0000","last_became_peered":"2024-09-16T15:07:22.088585+0000","last_unstale":"2024-09-16T15:07:22.773630+0000","last_undegraded":"2024-09-16T15:07:22.773630+0000","last_fullsized":"2024-09-16T15:07:22.773630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:19.377720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061723+0000","last_change":"2024-09-16T15:07:21.713440+0000","last_active":"2024-09-16T15:07:23.061723+0000","last_peered":"2024-09-16T15:07:23.061723+0000","last_clean":"2024-09-16T15:07:23.061723+0000","last_became_active":"2024-09-16T15:07:21.713127+0000","last_became_peered":"2024-09-16T15:07:21.713127+0000","last_unstale":"2024-09-16T15:07:23.061723+0000","last_undegraded":"2024-09-16T15:07:23.061723+0000","last_fullsized":"2024-09-16T15:07:23.061723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.239668+0000","last_change":"2024-09-16T15:07:17.671471+0000","last_active":"2024-09-16T15:07:27.239668+0000","last_peered":"2024-09-16T15:07:27.239668+0000","last_clean":"2024-09-16T15:07:27.239668+0000","last_became_active":"2024-09-16T15:07:15.656288+0000","last_became_peered":"2024-09-16T15:07:15.656288+0000","last_unstale":"2024-09-16T15:07:27.239668+0000","last_undegraded":"2024-09-16T15:07:27.239668+0000","last_fullsized":"2024-09-16T15:07:27.239668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:03.700310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026747759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773852+0000","last_change":"2024-09-16T15:07:22.087230+0000","last_active":"2024-09-16T15:07:22.773852+0000","last_peered":"2024-09-16T15:07:22.773852+0000","last_clean":"2024-09-16T15:07:22.773852+0000","last_became_active":"2024-09-16T15:07:22.086917+0000","last_became_peered":"2024-09-16T15:07:22.086917+0000","last_unstale":"2024-09-16T15:07:22.773852+0000","last_undegraded":"2024-09-16T15:07:22.773852+0000","last_fullsized":"2024-09-16T15:07:22.773852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:05.650468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061782+0000","last_change":"2024-09-16T15:07:21.727506+0000","last_active":"2024-09-16T15:07:23.061782+0000","last_peered":"2024-09-16T15:07:23.061782+0000","last_clean":"2024-09-16T15:07:23.061782+0000","last_became_active":"2024-09-16T15:07:21.724965+0000","last_became_peered":"2024-09-16T15:07:21.724965+0000","last_unstale":"2024-09-16T15:07:23.061782+0000","last_undegraded":"2024-09-16T15:07:23.061782+0000","last_fullsized":"2024-09-16T15:07:23.061782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:41.835469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.234823+0000","last_change":"2024-09-16T15:07:17.664973+0000","last_active":"2024-09-16T15:07:27.234823+0000","last_peered":"2024-09-16T15:07:27.234823+0000","last_clean":"2024-09-16T15:07:27.234823+0000","last_became_active":"2024-09-16T15:07:15.655047+0000","last_became_peered":"2024-09-16T15:07:15.655047+0000","last_unstale":"2024-09-16T15:07:27.234823+0000","last_undegraded":"2024-09-16T15:07:27.234823+0000","last_fullsized":"2024-09-16T15:07:27.234823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:53.308490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081824399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059410+0000","last_change":"2024-09-16T15:07:22.100811+0000","last_active":"2024-09-16T15:07:23.059410+0000","last_peered":"2024-09-16T15:07:23.059410+0000","last_clean":"2024-09-16T15:07:23.059410+0000","last_became_active":"2024-09-16T15:07:22.099732+0000","last_became_peered":"2024-09-16T15:07:22.099732+0000","last_unstale":"2024-09-16T15:07:23.059410+0000","last_undegraded":"2024-09-16T15:07:23.059410+0000","last_fullsized":"2024-09-16T15:07:23.059410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:36.011335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.057562+0000","last_change":"2024-09-16T15:07:21.733464+0000","last_active":"2024-09-16T15:07:23.057562+0000","last_peered":"2024-09-16T15:07:23.057562+0000","last_clean":"2024-09-16T15:07:23.057562+0000","last_became_active":"2024-09-16T15:07:21.732659+0000","last_became_peered":"2024-09-16T15:07:21.732659+0000","last_unstale":"2024-09-16T15:07:23.057562+0000","last_undegraded":"2024-09-16T15:07:23.057562+0000","last_fullsized":"2024-09-16T15:07:23.057562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.238873+0000","last_change":"2024-09-16T15:07:17.671161+0000","last_active":"2024-09-16T15:07:27.238873+0000","last_peered":"2024-09-16T15:07:27.238873+0000","last_clean":"2024-09-16T15:07:27.238873+0000","last_became_active":"2024-09-16T15:07:15.654237+0000","last_became_peered":"2024-09-16T15:07:15.654237+0000","last_unstale":"2024-09-16T15:07:27.238873+0000","last_undegraded":"2024-09-16T15:07:27.238873+0000","last_fullsized":"2024-09-16T15:07:27.238873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:56.780961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356239,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774462+0000","last_change":"2024-09-16T15:07:22.071750+0000","last_active":"2024-09-16T15:07:22.774462+0000","last_peered":"2024-09-16T15:07:22.774462+0000","last_clean":"2024-09-16T15:07:22.774462+0000","last_became_active":"2024-09-16T15:07:22.071469+0000","last_became_peered":"2024-09-16T15:07:22.071469+0000","last_unstale":"2024-09-16T15:07:22.774462+0000","last_undegraded":"2024-09-16T15:07:22.774462+0000","last_fullsized":"2024-09-16T15:07:22.774462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:45.906789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774639+0000","last_change":"2024-09-16T15:07:22.080700+0000","last_active":"2024-09-16T15:07:22.774639+0000","last_peered":"2024-09-16T15:07:22.774639+0000","last_clean":"2024-09-16T15:07:22.774639+0000","last_became_active":"2024-09-16T15:07:22.080300+0000","last_became_peered":"2024-09-16T15:07:22.080300+0000","last_unstale":"2024-09-16T15:07:22.774639+0000","last_undegraded":"2024-09-16T15:07:22.774639+0000","last_fullsized":"2024-09-16T15:07:22.774639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:27.703305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060850+0000","last_change":"2024-09-16T15:07:21.741959+0000","last_active":"2024-09-16T15:07:23.060850+0000","last_peered":"2024-09-16T15:07:23.060850+0000","last_clean":"2024-09-16T15:07:23.060850+0000","last_became_active":"2024-09-16T15:07:21.741329+0000","last_became_peered":"2024-09-16T15:07:21.741329+0000","last_unstale":"2024-09-16T15:07:23.060850+0000","last_undegraded":"2024-09-16T15:07:23.060850+0000","last_fullsized":"2024-09-16T15:07:23.060850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:31.395563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773674+0000","last_change":"2024-09-16T15:07:22.088939+0000","last_active":"2024-09-16T15:07:22.773674+0000","last_peered":"2024-09-16T15:07:22.773674+0000","last_clean":"2024-09-16T15:07:22.773674+0000","last_became_active":"2024-09-16T15:07:22.087096+0000","last_became_peered":"2024-09-16T15:07:22.087096+0000","last_unstale":"2024-09-16T15:07:22.773674+0000","last_undegraded":"2024-09-16T15:07:22.773674+0000","last_fullsized":"2024-09-16T15:07:22.773674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:49.184359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774748+0000","last_change":"2024-09-16T15:07:21.737806+0000","last_active":"2024-09-16T15:07:22.774748+0000","last_peered":"2024-09-16T15:07:22.774748+0000","last_clean":"2024-09-16T15:07:22.774748+0000","last_became_active":"2024-09-16T15:07:21.737289+0000","last_became_peered":"2024-09-16T15:07:21.737289+0000","last_unstale":"2024-09-16T15:07:22.774748+0000","last_undegraded":"2024-09-16T15:07:22.774748+0000","last_fullsized":"2024-09-16T15:07:22.774748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:15.691812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774875+0000","last_change":"2024-09-16T15:07:21.738040+0000","last_active":"2024-09-16T15:07:22.774875+0000","last_peered":"2024-09-16T15:07:22.774875+0000","last_clean":"2024-09-16T15:07:22.774875+0000","last_became_active":"2024-09-16T15:07:21.737276+0000","last_became_peered":"2024-09-16T15:07:21.737276+0000","last_unstale":"2024-09-16T15:07:22.774875+0000","last_undegraded":"2024-09-16T15:07:22.774875+0000","last_fullsized":"2024-09-16T15:07:22.774875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061179+0000","last_change":"2024-09-16T15:07:22.073415+0000","last_active":"2024-09-16T15:07:23.061179+0000","last_peered":"2024-09-16T15:07:23.061179+0000","last_clean":"2024-09-16T15:07:23.061179+0000","last_became_active":"2024-09-16T15:07:22.073160+0000","last_became_peered":"2024-09-16T15:07:22.073160+0000","last_unstale":"2024-09-16T15:07:23.061179+0000","last_undegraded":"2024-09-16T15:07:23.061179+0000","last_fullsized":"2024-09-16T15:07:23.061179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:32.398564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774167+0000","last_change":"2024-09-16T15:07:21.730081+0000","last_active":"2024-09-16T15:07:22.774167+0000","last_peered":"2024-09-16T15:07:22.774167+0000","last_clean":"2024-09-16T15:07:22.774167+0000","last_became_active":"2024-09-16T15:07:21.729450+0000","last_became_peered":"2024-09-16T15:07:21.729450+0000","last_unstale":"2024-09-16T15:07:22.774167+0000","last_undegraded":"2024-09-16T15:07:22.774167+0000","last_fullsized":"2024-09-16T15:07:22.774167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.058880+0000","last_change":"2024-09-16T15:07:22.100315+0000","last_active":"2024-09-16T15:07:23.058880+0000","last_peered":"2024-09-16T15:07:23.058880+0000","last_clean":"2024-09-16T15:07:23.058880+0000","last_became_active":"2024-09-16T15:07:22.099367+0000","last_became_peered":"2024-09-16T15:07:22.099367+0000","last_unstale":"2024-09-16T15:07:23.058880+0000","last_undegraded":"2024-09-16T15:07:23.058880+0000","last_fullsized":"2024-09-16T15:07:23.058880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:45.128492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061737+0000","last_change":"2024-09-16T15:07:22.079005+0000","last_active":"2024-09-16T15:07:23.061737+0000","last_peered":"2024-09-16T15:07:23.061737+0000","last_clean":"2024-09-16T15:07:23.061737+0000","last_became_active":"2024-09-16T15:07:22.077888+0000","last_became_peered":"2024-09-16T15:07:22.077888+0000","last_unstale":"2024-09-16T15:07:23.061737+0000","last_undegraded":"2024-09-16T15:07:23.061737+0000","last_fullsized":"2024-09-16T15:07:23.061737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:21.103662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059542+0000","last_change":"2024-09-16T15:07:21.724826+0000","last_active":"2024-09-16T15:07:23.059542+0000","last_peered":"2024-09-16T15:07:23.059542+0000","last_clean":"2024-09-16T15:07:23.059542+0000","last_became_active":"2024-09-16T15:07:21.724152+0000","last_became_peered":"2024-09-16T15:07:21.724152+0000","last_unstale":"2024-09-16T15:07:23.059542+0000","last_undegraded":"2024-09-16T15:07:23.059542+0000","last_fullsized":"2024-09-16T15:07:23.059542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:26.700305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062278+0000","last_change":"2024-09-16T15:07:22.087018+0000","last_active":"2024-09-16T15:07:23.062278+0000","last_peered":"2024-09-16T15:07:23.062278+0000","last_clean":"2024-09-16T15:07:23.062278+0000","last_became_active":"2024-09-16T15:07:22.086746+0000","last_became_peered":"2024-09-16T15:07:22.086746+0000","last_unstale":"2024-09-16T15:07:23.062278+0000","last_undegraded":"2024-09-16T15:07:23.062278+0000","last_fullsized":"2024-09-16T15:07:23.062278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:11.167685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773475+0000","last_change":"2024-09-16T15:07:21.726511+0000","last_active":"2024-09-16T15:07:22.773475+0000","last_peered":"2024-09-16T15:07:22.773475+0000","last_clean":"2024-09-16T15:07:22.773475+0000","last_became_active":"2024-09-16T15:07:21.725971+0000","last_became_peered":"2024-09-16T15:07:21.725971+0000","last_unstale":"2024-09-16T15:07:22.773475+0000","last_undegraded":"2024-09-16T15:07:22.773475+0000","last_fullsized":"2024-09-16T15:07:22.773475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:15.691812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061171+0000","last_change":"2024-09-16T15:07:22.093131+0000","last_active":"2024-09-16T15:07:23.061171+0000","last_peered":"2024-09-16T15:07:23.061171+0000","last_clean":"2024-09-16T15:07:23.061171+0000","last_became_active":"2024-09-16T15:07:22.092810+0000","last_became_peered":"2024-09-16T15:07:22.092810+0000","last_unstale":"2024-09-16T15:07:23.061171+0000","last_undegraded":"2024-09-16T15:07:23.061171+0000","last_fullsized":"2024-09-16T15:07:23.061171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:27.872651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773764+0000","last_change":"2024-09-16T15:07:21.709700+0000","last_active":"2024-09-16T15:07:22.773764+0000","last_peered":"2024-09-16T15:07:22.773764+0000","last_clean":"2024-09-16T15:07:22.773764+0000","last_became_active":"2024-09-16T15:07:21.708187+0000","last_became_peered":"2024-09-16T15:07:21.708187+0000","last_unstale":"2024-09-16T15:07:22.773764+0000","last_undegraded":"2024-09-16T15:07:22.773764+0000","last_fullsized":"2024-09-16T15:07:22.773764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:50.814161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061551+0000","last_change":"2024-09-16T15:07:22.067775+0000","last_active":"2024-09-16T15:07:23.061551+0000","last_peered":"2024-09-16T15:07:23.061551+0000","last_clean":"2024-09-16T15:07:23.061551+0000","last_became_active":"2024-09-16T15:07:22.067259+0000","last_became_peered":"2024-09-16T15:07:22.067259+0000","last_unstale":"2024-09-16T15:07:23.061551+0000","last_undegraded":"2024-09-16T15:07:23.061551+0000","last_fullsized":"2024-09-16T15:07:23.061551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:04.412982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060444+0000","last_change":"2024-09-16T15:07:21.726917+0000","last_active":"2024-09-16T15:07:23.060444+0000","last_peered":"2024-09-16T15:07:23.060444+0000","last_clean":"2024-09-16T15:07:23.060444+0000","last_became_active":"2024-09-16T15:07:21.724806+0000","last_became_peered":"2024-09-16T15:07:21.724806+0000","last_unstale":"2024-09-16T15:07:23.060444+0000","last_undegraded":"2024-09-16T15:07:23.060444+0000","last_fullsized":"2024-09-16T15:07:23.060444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:54.201635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774618+0000","last_change":"2024-09-16T15:07:22.059433+0000","last_active":"2024-09-16T15:07:22.774618+0000","last_peered":"2024-09-16T15:07:22.774618+0000","last_clean":"2024-09-16T15:07:22.774618+0000","last_became_active":"2024-09-16T15:07:22.058973+0000","last_became_peered":"2024-09-16T15:07:22.058973+0000","last_unstale":"2024-09-16T15:07:22.774618+0000","last_undegraded":"2024-09-16T15:07:22.774618+0000","last_fullsized":"2024-09-16T15:07:22.774618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:11.167685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773963+0000","last_change":"2024-09-16T15:07:21.726761+0000","last_active":"2024-09-16T15:07:22.773963+0000","last_peered":"2024-09-16T15:07:22.773963+0000","last_clean":"2024-09-16T15:07:22.773963+0000","last_became_active":"2024-09-16T15:07:21.726186+0000","last_became_peered":"2024-09-16T15:07:21.726186+0000","last_unstale":"2024-09-16T15:07:22.773963+0000","last_undegraded":"2024-09-16T15:07:22.773963+0000","last_fullsized":"2024-09-16T15:07:22.773963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774571+0000","last_change":"2024-09-16T15:07:22.059781+0000","last_active":"2024-09-16T15:07:22.774571+0000","last_peered":"2024-09-16T15:07:22.774571+0000","last_clean":"2024-09-16T15:07:22.774571+0000","last_became_active":"2024-09-16T15:07:22.059423+0000","last_became_peered":"2024-09-16T15:07:22.059423+0000","last_unstale":"2024-09-16T15:07:22.774571+0000","last_undegraded":"2024-09-16T15:07:22.774571+0000","last_fullsized":"2024-09-16T15:07:22.774571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:22.080488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061523+0000","last_change":"2024-09-16T15:07:21.743147+0000","last_active":"2024-09-16T15:07:23.061523+0000","last_peered":"2024-09-16T15:07:23.061523+0000","last_clean":"2024-09-16T15:07:23.061523+0000","last_became_active":"2024-09-16T15:07:21.742365+0000","last_became_peered":"2024-09-16T15:07:21.742365+0000","last_unstale":"2024-09-16T15:07:23.061523+0000","last_undegraded":"2024-09-16T15:07:23.061523+0000","last_fullsized":"2024-09-16T15:07:23.061523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:10.164684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060096+0000","last_change":"2024-09-16T15:07:22.076343+0000","last_active":"2024-09-16T15:07:23.060096+0000","last_peered":"2024-09-16T15:07:23.060096+0000","last_clean":"2024-09-16T15:07:23.060096+0000","last_became_active":"2024-09-16T15:07:22.075982+0000","last_became_peered":"2024-09-16T15:07:22.075982+0000","last_unstale":"2024-09-16T15:07:23.060096+0000","last_undegraded":"2024-09-16T15:07:23.060096+0000","last_fullsized":"2024-09-16T15:07:23.060096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:25:32.065204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062410+0000","last_change":"2024-09-16T15:07:21.722589+0000","last_active":"2024-09-16T15:07:23.062410+0000","last_peered":"2024-09-16T15:07:23.062410+0000","last_clean":"2024-09-16T15:07:23.062410+0000","last_became_active":"2024-09-16T15:07:21.722226+0000","last_became_peered":"2024-09-16T15:07:21.722226+0000","last_unstale":"2024-09-16T15:07:23.062410+0000","last_undegraded":"2024-09-16T15:07:23.062410+0000","last_fullsized":"2024-09-16T15:07:23.062410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774405+0000","last_change":"2024-09-16T15:07:22.061731+0000","last_active":"2024-09-16T15:07:22.774405+0000","last_peered":"2024-09-16T15:07:22.774405+0000","last_clean":"2024-09-16T15:07:22.774405+0000","last_became_active":"2024-09-16T15:07:22.061419+0000","last_became_peered":"2024-09-16T15:07:22.061419+0000","last_unstale":"2024-09-16T15:07:22.774405+0000","last_undegraded":"2024-09-16T15:07:22.774405+0000","last_fullsized":"2024-09-16T15:07:22.774405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:47.569148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774142+0000","last_change":"2024-09-16T15:07:21.709152+0000","last_active":"2024-09-16T15:07:22.774142+0000","last_peered":"2024-09-16T15:07:22.774142+0000","last_clean":"2024-09-16T15:07:22.774142+0000","last_became_active":"2024-09-16T15:07:21.708468+0000","last_became_peered":"2024-09-16T15:07:21.708468+0000","last_unstale":"2024-09-16T15:07:22.774142+0000","last_undegraded":"2024-09-16T15:07:22.774142+0000","last_fullsized":"2024-09-16T15:07:22.774142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":69541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":659842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":658806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":68365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:07:29.463 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:07:29.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.602+0000 7efd298a1640 1 Processor -- start 2024-09-16T15:07:29.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.602+0000 7efd298a1640 1 -- start start 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.602+0000 7efd298a1640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efd240a3980 0x7efd2406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.602+0000 7efd298a1640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.602+0000 7efd298a1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd240a66c0 0x7efd24143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.602+0000 7efd298a1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd240a57a0 con 0x7efd240a66c0 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.602+0000 7efd298a1640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd241438e0 con 0x7efd240a3980 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.602+0000 7efd298a1640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd240a3d60 con 0x7efd2406ae30 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.602+0000 7efd227fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd237fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efd240a66c0 0x7efd24143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd227fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd240a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53046/0 (socket says 172.21.15.133:53046) 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd227fc640 1 -- 172.21.15.133:0/3245515101 learned_addr learned my addr 172.21.15.133:0/3245515101 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:29.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd22ffd640 1 --2- 172.21.15.133:0/3245515101 >> v2:172.21.15.191:3300/0 conn(0x7efd240a3980 0x7efd2406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd227fc640 1 -- 172.21.15.133:0/3245515101 >> v2:172.21.15.191:3300/0 conn(0x7efd240a3980 msgr2=0x7efd2406a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd227fc640 1 --2- 172.21.15.133:0/3245515101 >> v2:172.21.15.191:3300/0 conn(0x7efd240a3980 0x7efd2406a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd227fc640 1 -- 172.21.15.133:0/3245515101 >> v2:172.21.15.133:3300/0 conn(0x7efd240a66c0 msgr2=0x7efd24143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd227fc640 1 --2- 172.21.15.133:0/3245515101 >> v2:172.21.15.133:3300/0 conn(0x7efd240a66c0 0x7efd24143ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd227fc640 1 -- 172.21.15.133:0/3245515101 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd241435f0 con 0x7efd2406ae30 2024-09-16T15:07:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd22ffd640 1 --2- 172.21.15.133:0/3245515101 >> v2:172.21.15.191:3300/0 conn(0x7efd240a3980 0x7efd2406a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:29.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd237fe640 1 --2- 172.21.15.133:0/3245515101 >> v2:172.21.15.133:3300/0 conn(0x7efd240a66c0 0x7efd24143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:29.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd227fc640 1 --2- 172.21.15.133:0/3245515101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd240a6180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efd240683d0 tx=0x7efd0c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f8fdb363cfacc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd21ffb640 1 -- 172.21.15.133:0/3245515101 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c0ab650 con 0x7efd2406ae30 2024-09-16T15:07:29.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd21ffb640 1 -- 172.21.15.133:0/3245515101 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c0ab7b0 con 0x7efd2406ae30 2024-09-16T15:07:29.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd21ffb640 1 -- 172.21.15.133:0/3245515101 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c0abb40 con 0x7efd2406ae30 2024-09-16T15:07:29.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd298a1640 1 -- 172.21.15.133:0/3245515101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 msgr2=0x7efd240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd298a1640 1 --2- 172.21.15.133:0/3245515101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd240a6180 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efd240683d0 tx=0x7efd0c01c470 comp rx=0 tx=0).stop 2024-09-16T15:07:29.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd298a1640 1 -- 172.21.15.133:0/3245515101 shutdown_connections 2024-09-16T15:07:29.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd298a1640 1 --2- 172.21.15.133:0/3245515101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd240a6180 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd298a1640 1 --2- 172.21.15.133:0/3245515101 >> v2:172.21.15.133:3300/0 conn(0x7efd240a66c0 0x7efd24143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd298a1640 1 --2- 172.21.15.133:0/3245515101 >> v2:172.21.15.191:3300/0 conn(0x7efd240a3980 0x7efd2406a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd298a1640 1 -- 172.21.15.133:0/3245515101 >> 172.21.15.133:0/3245515101 conn(0x7efd240a3fd0 msgr2=0x7efd240a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd298a1640 1 -- 172.21.15.133:0/3245515101 shutdown_connections 2024-09-16T15:07:29.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.606+0000 7efd298a1640 1 -- 172.21.15.133:0/3245515101 wait complete. 2024-09-16T15:07:29.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 Processor -- start 2024-09-16T15:07:29.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 -- start start 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd24148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd240a3980 0x7efd24149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd240a66c0 0x7efd2414a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd24143210 con 0x7efd240a3980 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2409a5e0 con 0x7efd240a66c0 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2409a880 con 0x7efd2406ae30 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd22ffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd24148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd227fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd240a3980 0x7efd24149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd22ffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd24148790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53050/0 (socket says 172.21.15.133:53050) 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd22ffd640 1 -- 172.21.15.133:0/3457183912 learned_addr learned my addr 172.21.15.133:0/3457183912 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd237fe640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd240a66c0 0x7efd2414a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd22ffd640 1 -- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd240a66c0 msgr2=0x7efd2414a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd22ffd640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd240a66c0 0x7efd2414a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd22ffd640 1 -- 172.21.15.133:0/3457183912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd240a3980 msgr2=0x7efd24149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd22ffd640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd240a3980 0x7efd24149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd22ffd640 1 -- 172.21.15.133:0/3457183912 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd14009420 con 0x7efd2406ae30 2024-09-16T15:07:29.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd227fc640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd240a3980 0x7efd24149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:29.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd237fe640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd240a66c0 0x7efd2414a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:29.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd22ffd640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd24148790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00cf60 tx=0x7efd1c00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd1bfff640 1 -- 172.21.15.133:0/3457183912 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c00d840 con 0x7efd2406ae30 2024-09-16T15:07:29.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd1bfff640 1 -- 172.21.15.133:0/3457183912 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c00d9a0 con 0x7efd2406ae30 2024-09-16T15:07:29.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 -- 172.21.15.133:0/3457183912 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0c0094d0 con 0x7efd2406ae30 2024-09-16T15:07:29.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd1bfff640 1 -- 172.21.15.133:0/3457183912 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c00dd30 con 0x7efd2406ae30 2024-09-16T15:07:29.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 -- 172.21.15.133:0/3457183912 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd24149650 con 0x7efd2406ae30 2024-09-16T15:07:29.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd298a1640 1 -- 172.21.15.133:0/3457183912 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd24143380 con 0x7efd2406ae30 2024-09-16T15:07:29.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd1bfff640 1 -- 172.21.15.133:0/3457183912 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7efd1c015440 con 0x7efd2406ae30 2024-09-16T15:07:29.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.610+0000 7efd1bfff640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:29.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.614+0000 7efd1bfff640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:29.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.614+0000 7efd1bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:29.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.614+0000 7efd1bfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:29.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.614+0000 7efd1bfff640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7efd04097840 0x7efd04099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.614+0000 7efd1bfff640 1 -- 172.21.15.133:0/3457183912 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7efd1c0c0cb0 con 0x7efd2406ae30 2024-09-16T15:07:29.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.614+0000 7efd227fc640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7efd04097840 0x7efd04099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.614+0000 7efd227fc640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7efd04097840 0x7efd04099d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0af000 tx=0x7efd0c0b6040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.614+0000 7efd1bfff640 1 -- 172.21.15.133:0/3457183912 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efd1c08e470 con 0x7efd2406ae30 2024-09-16T15:07:29.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.758+0000 7efd298a1640 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2024-09-16T15:07:29.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.758+0000 7efd298a1640 1 -- 172.21.15.133:0/3457183912 --> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd24143380 con 0x7efd04097840 2024-09-16T15:07:29.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd1bfff640 1 -- 172.21.15.133:0/3457183912 <== mgr.4102 v2:172.21.15.191:6824/3926506186 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+271531 (secure 0 0 0) 0x7efd24143380 con 0x7efd04097840 2024-09-16T15:07:29.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd1bfff640 20 mgrc handle_command_reply tid 0 r 0 2024-09-16T15:07:29.786 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:29.787 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 10 mgrc shutdown 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 -- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7efd04097840 msgr2=0x7efd04099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7efd04097840 0x7efd04099d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0af000 tx=0x7efd0c0b6040 comp rx=0 tx=0).stop 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 -- 172.21.15.133:0/3457183912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 msgr2=0x7efd24148790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd24148790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00cf60 tx=0x7efd1c00cf90 comp rx=0 tx=0).stop 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 -- 172.21.15.133:0/3457183912 shutdown_connections 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd240a66c0 0x7efd2414a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7efd04097840 0x7efd04099d00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd240a3980 0x7efd24149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 --2- 172.21.15.133:0/3457183912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd2406ae30 0x7efd24148790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 -- 172.21.15.133:0/3457183912 >> 172.21.15.133:0/3457183912 conn(0x7efd240a3fd0 msgr2=0x7efd240a2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 -- 172.21.15.133:0/3457183912 shutdown_connections 2024-09-16T15:07:29.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.782+0000 7efd298a1640 1 -- 172.21.15.133:0/3457183912 wait complete. 2024-09-16T15:07:29.801 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-16T15:07:28.691398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592749,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110084,"kb_used_data":1984,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377377276,"statfs":{"total":386547056640,"available":386434330624,"internally_reserved":0,"allocated":2031616,"data_stored":1456554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6374,"internal_metadata":110618394},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.640220"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774982+0000","last_change":"2024-09-16T15:07:22.073492+0000","last_active":"2024-09-16T15:07:22.774982+0000","last_peered":"2024-09-16T15:07:22.774982+0000","last_clean":"2024-09-16T15:07:22.774982+0000","last_became_active":"2024-09-16T15:07:22.073186+0000","last_became_peered":"2024-09-16T15:07:22.073186+0000","last_unstale":"2024-09-16T15:07:22.774982+0000","last_undegraded":"2024-09-16T15:07:22.774982+0000","last_fullsized":"2024-09-16T15:07:22.774982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774795+0000","last_change":"2024-09-16T15:07:22.089956+0000","last_active":"2024-09-16T15:07:22.774795+0000","last_peered":"2024-09-16T15:07:22.774795+0000","last_clean":"2024-09-16T15:07:22.774795+0000","last_became_active":"2024-09-16T15:07:22.089659+0000","last_became_peered":"2024-09-16T15:07:22.089659+0000","last_unstale":"2024-09-16T15:07:22.774795+0000","last_undegraded":"2024-09-16T15:07:22.774795+0000","last_fullsized":"2024-09-16T15:07:22.774795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:14.566411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060926+0000","last_change":"2024-09-16T15:07:22.092432+0000","last_active":"2024-09-16T15:07:23.060926+0000","last_peered":"2024-09-16T15:07:23.060926+0000","last_clean":"2024-09-16T15:07:23.060926+0000","last_became_active":"2024-09-16T15:07:22.092188+0000","last_became_peered":"2024-09-16T15:07:22.092188+0000","last_unstale":"2024-09-16T15:07:23.060926+0000","last_undegraded":"2024-09-16T15:07:23.060926+0000","last_fullsized":"2024-09-16T15:07:23.060926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:25.997418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773890+0000","last_change":"2024-09-16T15:07:22.095720+0000","last_active":"2024-09-16T15:07:22.773890+0000","last_peered":"2024-09-16T15:07:22.773890+0000","last_clean":"2024-09-16T15:07:22.773890+0000","last_became_active":"2024-09-16T15:07:22.093626+0000","last_became_peered":"2024-09-16T15:07:22.093626+0000","last_unstale":"2024-09-16T15:07:22.773890+0000","last_undegraded":"2024-09-16T15:07:22.773890+0000","last_fullsized":"2024-09-16T15:07:22.773890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:24.351732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060209+0000","last_change":"2024-09-16T15:07:22.077679+0000","last_active":"2024-09-16T15:07:23.060209+0000","last_peered":"2024-09-16T15:07:23.060209+0000","last_clean":"2024-09-16T15:07:23.060209+0000","last_became_active":"2024-09-16T15:07:22.077387+0000","last_became_peered":"2024-09-16T15:07:22.077387+0000","last_unstale":"2024-09-16T15:07:23.060209+0000","last_undegraded":"2024-09-16T15:07:23.060209+0000","last_fullsized":"2024-09-16T15:07:23.060209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:09.497997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059444+0000","last_change":"2024-09-16T15:07:22.077568+0000","last_active":"2024-09-16T15:07:23.059444+0000","last_peered":"2024-09-16T15:07:23.059444+0000","last_clean":"2024-09-16T15:07:23.059444+0000","last_became_active":"2024-09-16T15:07:22.076638+0000","last_became_peered":"2024-09-16T15:07:22.076638+0000","last_unstale":"2024-09-16T15:07:23.059444+0000","last_undegraded":"2024-09-16T15:07:23.059444+0000","last_fullsized":"2024-09-16T15:07:23.059444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:04:28.264456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.058994+0000","last_change":"2024-09-16T15:07:22.075898+0000","last_active":"2024-09-16T15:07:23.058994+0000","last_peered":"2024-09-16T15:07:23.058994+0000","last_clean":"2024-09-16T15:07:23.058994+0000","last_became_active":"2024-09-16T15:07:22.075448+0000","last_became_peered":"2024-09-16T15:07:22.075448+0000","last_unstale":"2024-09-16T15:07:23.058994+0000","last_undegraded":"2024-09-16T15:07:23.058994+0000","last_fullsized":"2024-09-16T15:07:23.058994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:03.093921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773305+0000","last_change":"2024-09-16T15:07:22.096662+0000","last_active":"2024-09-16T15:07:22.773305+0000","last_peered":"2024-09-16T15:07:22.773305+0000","last_clean":"2024-09-16T15:07:22.773305+0000","last_became_active":"2024-09-16T15:07:22.095671+0000","last_became_peered":"2024-09-16T15:07:22.095671+0000","last_unstale":"2024-09-16T15:07:22.773305+0000","last_undegraded":"2024-09-16T15:07:22.773305+0000","last_fullsized":"2024-09-16T15:07:22.773305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:47.108958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062345+0000","last_change":"2024-09-16T15:07:22.092184+0000","last_active":"2024-09-16T15:07:23.062345+0000","last_peered":"2024-09-16T15:07:23.062345+0000","last_clean":"2024-09-16T15:07:23.062345+0000","last_became_active":"2024-09-16T15:07:22.091896+0000","last_became_peered":"2024-09-16T15:07:22.091896+0000","last_unstale":"2024-09-16T15:07:23.062345+0000","last_undegraded":"2024-09-16T15:07:23.062345+0000","last_fullsized":"2024-09-16T15:07:23.062345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774013+0000","last_change":"2024-09-16T15:07:22.093487+0000","last_active":"2024-09-16T15:07:22.774013+0000","last_peered":"2024-09-16T15:07:22.774013+0000","last_clean":"2024-09-16T15:07:22.774013+0000","last_became_active":"2024-09-16T15:07:22.091134+0000","last_became_peered":"2024-09-16T15:07:22.091134+0000","last_unstale":"2024-09-16T15:07:22.774013+0000","last_undegraded":"2024-09-16T15:07:22.774013+0000","last_fullsized":"2024-09-16T15:07:22.774013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:55.259949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773989+0000","last_change":"2024-09-16T15:07:22.091123+0000","last_active":"2024-09-16T15:07:22.773989+0000","last_peered":"2024-09-16T15:07:22.773989+0000","last_clean":"2024-09-16T15:07:22.773989+0000","last_became_active":"2024-09-16T15:07:22.090822+0000","last_became_peered":"2024-09-16T15:07:22.090822+0000","last_unstale":"2024-09-16T15:07:22.773989+0000","last_undegraded":"2024-09-16T15:07:22.773989+0000","last_fullsized":"2024-09-16T15:07:22.773989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:41:09.496808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774722+0000","last_change":"2024-09-16T15:07:22.075137+0000","last_active":"2024-09-16T15:07:22.774722+0000","last_peered":"2024-09-16T15:07:22.774722+0000","last_clean":"2024-09-16T15:07:22.774722+0000","last_became_active":"2024-09-16T15:07:22.074779+0000","last_became_peered":"2024-09-16T15:07:22.074779+0000","last_unstale":"2024-09-16T15:07:22.774722+0000","last_undegraded":"2024-09-16T15:07:22.774722+0000","last_fullsized":"2024-09-16T15:07:22.774722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:34.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061606+0000","last_change":"2024-09-16T15:07:22.069681+0000","last_active":"2024-09-16T15:07:23.061606+0000","last_peered":"2024-09-16T15:07:23.061606+0000","last_clean":"2024-09-16T15:07:23.061606+0000","last_became_active":"2024-09-16T15:07:22.069331+0000","last_became_peered":"2024-09-16T15:07:22.069331+0000","last_unstale":"2024-09-16T15:07:23.061606+0000","last_undegraded":"2024-09-16T15:07:23.061606+0000","last_fullsized":"2024-09-16T15:07:23.061606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:27.703305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773541+0000","last_change":"2024-09-16T15:07:22.093463+0000","last_active":"2024-09-16T15:07:22.773541+0000","last_peered":"2024-09-16T15:07:22.773541+0000","last_clean":"2024-09-16T15:07:22.773541+0000","last_became_active":"2024-09-16T15:07:22.091154+0000","last_became_peered":"2024-09-16T15:07:22.091154+0000","last_unstale":"2024-09-16T15:07:22.773541+0000","last_undegraded":"2024-09-16T15:07:22.773541+0000","last_fullsized":"2024-09-16T15:07:22.773541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:24:09.086333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773431+0000","last_change":"2024-09-16T15:07:22.087589+0000","last_active":"2024-09-16T15:07:22.773431+0000","last_peered":"2024-09-16T15:07:22.773431+0000","last_clean":"2024-09-16T15:07:22.773431+0000","last_became_active":"2024-09-16T15:07:22.087167+0000","last_became_peered":"2024-09-16T15:07:22.087167+0000","last_unstale":"2024-09-16T15:07:22.773431+0000","last_undegraded":"2024-09-16T15:07:22.773431+0000","last_fullsized":"2024-09-16T15:07:22.773431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:26.349524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.058848+0000","last_change":"2024-09-16T15:07:22.076065+0000","last_active":"2024-09-16T15:07:23.058848+0000","last_peered":"2024-09-16T15:07:23.058848+0000","last_clean":"2024-09-16T15:07:23.058848+0000","last_became_active":"2024-09-16T15:07:22.075724+0000","last_became_peered":"2024-09-16T15:07:22.075724+0000","last_unstale":"2024-09-16T15:07:23.058848+0000","last_undegraded":"2024-09-16T15:07:23.058848+0000","last_fullsized":"2024-09-16T15:07:23.058848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:20:30.555559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774060+0000","last_change":"2024-09-16T15:07:22.091007+0000","last_active":"2024-09-16T15:07:22.774060+0000","last_peered":"2024-09-16T15:07:22.774060+0000","last_clean":"2024-09-16T15:07:22.774060+0000","last_became_active":"2024-09-16T15:07:22.089216+0000","last_became_peered":"2024-09-16T15:07:22.089216+0000","last_unstale":"2024-09-16T15:07:22.774060+0000","last_undegraded":"2024-09-16T15:07:22.774060+0000","last_fullsized":"2024-09-16T15:07:22.774060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:43.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061475+0000","last_change":"2024-09-16T15:07:22.077822+0000","last_active":"2024-09-16T15:07:23.061475+0000","last_peered":"2024-09-16T15:07:23.061475+0000","last_clean":"2024-09-16T15:07:23.061475+0000","last_became_active":"2024-09-16T15:07:22.076545+0000","last_became_peered":"2024-09-16T15:07:22.076545+0000","last_unstale":"2024-09-16T15:07:23.061475+0000","last_undegraded":"2024-09-16T15:07:23.061475+0000","last_fullsized":"2024-09-16T15:07:23.061475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:44:03.892835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061811+0000","last_change":"2024-09-16T15:07:22.074503+0000","last_active":"2024-09-16T15:07:23.061811+0000","last_peered":"2024-09-16T15:07:23.061811+0000","last_clean":"2024-09-16T15:07:23.061811+0000","last_became_active":"2024-09-16T15:07:22.074240+0000","last_became_peered":"2024-09-16T15:07:22.074240+0000","last_unstale":"2024-09-16T15:07:23.061811+0000","last_undegraded":"2024-09-16T15:07:23.061811+0000","last_fullsized":"2024-09-16T15:07:23.061811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:45.906789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773917+0000","last_change":"2024-09-16T15:07:22.090412+0000","last_active":"2024-09-16T15:07:22.773917+0000","last_peered":"2024-09-16T15:07:22.773917+0000","last_clean":"2024-09-16T15:07:22.773917+0000","last_became_active":"2024-09-16T15:07:22.089178+0000","last_became_peered":"2024-09-16T15:07:22.089178+0000","last_unstale":"2024-09-16T15:07:22.773917+0000","last_undegraded":"2024-09-16T15:07:22.773917+0000","last_fullsized":"2024-09-16T15:07:22.773917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:20.559503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773814+0000","last_change":"2024-09-16T15:07:22.085622+0000","last_active":"2024-09-16T15:07:22.773814+0000","last_peered":"2024-09-16T15:07:22.773814+0000","last_clean":"2024-09-16T15:07:22.773814+0000","last_became_active":"2024-09-16T15:07:22.085321+0000","last_became_peered":"2024-09-16T15:07:22.085321+0000","last_unstale":"2024-09-16T15:07:22.773814+0000","last_undegraded":"2024-09-16T15:07:22.773814+0000","last_fullsized":"2024-09-16T15:07:22.773814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:25.939312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773588+0000","last_change":"2024-09-16T15:07:22.088691+0000","last_active":"2024-09-16T15:07:22.773588+0000","last_peered":"2024-09-16T15:07:22.773588+0000","last_clean":"2024-09-16T15:07:22.773588+0000","last_became_active":"2024-09-16T15:07:22.085505+0000","last_became_peered":"2024-09-16T15:07:22.085505+0000","last_unstale":"2024-09-16T15:07:22.773588+0000","last_undegraded":"2024-09-16T15:07:22.773588+0000","last_fullsized":"2024-09-16T15:07:22.773588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:20.387713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059028+0000","last_change":"2024-09-16T15:07:22.093287+0000","last_active":"2024-09-16T15:07:23.059028+0000","last_peered":"2024-09-16T15:07:23.059028+0000","last_clean":"2024-09-16T15:07:23.059028+0000","last_became_active":"2024-09-16T15:07:22.092983+0000","last_became_peered":"2024-09-16T15:07:22.092983+0000","last_unstale":"2024-09-16T15:07:23.059028+0000","last_undegraded":"2024-09-16T15:07:23.059028+0000","last_fullsized":"2024-09-16T15:07:23.059028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:32.532671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773341+0000","last_change":"2024-09-16T15:07:22.096232+0000","last_active":"2024-09-16T15:07:22.773341+0000","last_peered":"2024-09-16T15:07:22.773341+0000","last_clean":"2024-09-16T15:07:22.773341+0000","last_became_active":"2024-09-16T15:07:22.091234+0000","last_became_peered":"2024-09-16T15:07:22.091234+0000","last_unstale":"2024-09-16T15:07:22.773341+0000","last_undegraded":"2024-09-16T15:07:22.773341+0000","last_fullsized":"2024-09-16T15:07:22.773341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:23.826942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774094+0000","last_change":"2024-09-16T15:07:22.088597+0000","last_active":"2024-09-16T15:07:22.774094+0000","last_peered":"2024-09-16T15:07:22.774094+0000","last_clean":"2024-09-16T15:07:22.774094+0000","last_became_active":"2024-09-16T15:07:22.086951+0000","last_became_peered":"2024-09-16T15:07:22.086951+0000","last_unstale":"2024-09-16T15:07:22.774094+0000","last_undegraded":"2024-09-16T15:07:22.774094+0000","last_fullsized":"2024-09-16T15:07:22.774094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:57.389973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062294+0000","last_change":"2024-09-16T15:07:22.067847+0000","last_active":"2024-09-16T15:07:23.062294+0000","last_peered":"2024-09-16T15:07:23.062294+0000","last_clean":"2024-09-16T15:07:23.062294+0000","last_became_active":"2024-09-16T15:07:22.067337+0000","last_became_peered":"2024-09-16T15:07:22.067337+0000","last_unstale":"2024-09-16T15:07:23.062294+0000","last_undegraded":"2024-09-16T15:07:23.062294+0000","last_fullsized":"2024-09-16T15:07:23.062294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:23.826942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061869+0000","last_change":"2024-09-16T15:07:22.079269+0000","last_active":"2024-09-16T15:07:23.061869+0000","last_peered":"2024-09-16T15:07:23.061869+0000","last_clean":"2024-09-16T15:07:23.061869+0000","last_became_active":"2024-09-16T15:07:22.078377+0000","last_became_peered":"2024-09-16T15:07:22.078377+0000","last_unstale":"2024-09-16T15:07:23.061869+0000","last_undegraded":"2024-09-16T15:07:23.061869+0000","last_fullsized":"2024-09-16T15:07:23.061869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:51.357090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061889+0000","last_change":"2024-09-16T15:07:22.067944+0000","last_active":"2024-09-16T15:07:23.061889+0000","last_peered":"2024-09-16T15:07:23.061889+0000","last_clean":"2024-09-16T15:07:23.061889+0000","last_became_active":"2024-09-16T15:07:22.067670+0000","last_became_peered":"2024-09-16T15:07:22.067670+0000","last_unstale":"2024-09-16T15:07:23.061889+0000","last_undegraded":"2024-09-16T15:07:23.061889+0000","last_fullsized":"2024-09-16T15:07:23.061889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:10.707281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061839+0000","last_change":"2024-09-16T15:07:22.074211+0000","last_active":"2024-09-16T15:07:23.061839+0000","last_peered":"2024-09-16T15:07:23.061839+0000","last_clean":"2024-09-16T15:07:23.061839+0000","last_became_active":"2024-09-16T15:07:22.073954+0000","last_became_peered":"2024-09-16T15:07:22.073954+0000","last_unstale":"2024-09-16T15:07:23.061839+0000","last_undegraded":"2024-09-16T15:07:23.061839+0000","last_fullsized":"2024-09-16T15:07:23.061839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:51.922251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773716+0000","last_change":"2024-09-16T15:07:22.088989+0000","last_active":"2024-09-16T15:07:22.773716+0000","last_peered":"2024-09-16T15:07:22.773716+0000","last_clean":"2024-09-16T15:07:22.773716+0000","last_became_active":"2024-09-16T15:07:22.085895+0000","last_became_peered":"2024-09-16T15:07:22.085895+0000","last_unstale":"2024-09-16T15:07:22.773716+0000","last_undegraded":"2024-09-16T15:07:22.773716+0000","last_fullsized":"2024-09-16T15:07:22.773716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:58.765327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060913+0000","last_change":"2024-09-16T15:07:22.092307+0000","last_active":"2024-09-16T15:07:23.060913+0000","last_peered":"2024-09-16T15:07:23.060913+0000","last_clean":"2024-09-16T15:07:23.060913+0000","last_became_active":"2024-09-16T15:07:22.091941+0000","last_became_peered":"2024-09-16T15:07:22.091941+0000","last_unstale":"2024-09-16T15:07:23.060913+0000","last_undegraded":"2024-09-16T15:07:23.060913+0000","last_fullsized":"2024-09-16T15:07:23.060913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:14.566411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773367+0000","last_change":"2024-09-16T15:07:22.096217+0000","last_active":"2024-09-16T15:07:22.773367+0000","last_peered":"2024-09-16T15:07:22.773367+0000","last_clean":"2024-09-16T15:07:22.773367+0000","last_became_active":"2024-09-16T15:07:22.092983+0000","last_became_peered":"2024-09-16T15:07:22.092983+0000","last_unstale":"2024-09-16T15:07:22.773367+0000","last_undegraded":"2024-09-16T15:07:22.773367+0000","last_fullsized":"2024-09-16T15:07:22.773367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:04.412982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060607+0000","last_change":"2024-09-16T15:07:22.079554+0000","last_active":"2024-09-16T15:07:23.060607+0000","last_peered":"2024-09-16T15:07:23.060607+0000","last_clean":"2024-09-16T15:07:23.060607+0000","last_became_active":"2024-09-16T15:07:22.078495+0000","last_became_peered":"2024-09-16T15:07:22.078495+0000","last_unstale":"2024-09-16T15:07:23.060607+0000","last_undegraded":"2024-09-16T15:07:23.060607+0000","last_fullsized":"2024-09-16T15:07:23.060607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:25.939312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060889+0000","last_change":"2024-09-16T15:07:21.742735+0000","last_active":"2024-09-16T15:07:23.060889+0000","last_peered":"2024-09-16T15:07:23.060889+0000","last_clean":"2024-09-16T15:07:23.060889+0000","last_became_active":"2024-09-16T15:07:21.742323+0000","last_became_peered":"2024-09-16T15:07:21.742323+0000","last_unstale":"2024-09-16T15:07:23.060889+0000","last_undegraded":"2024-09-16T15:07:23.060889+0000","last_fullsized":"2024-09-16T15:07:23.060889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:09.704280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061565+0000","last_change":"2024-09-16T15:07:22.079328+0000","last_active":"2024-09-16T15:07:23.061565+0000","last_peered":"2024-09-16T15:07:23.061565+0000","last_clean":"2024-09-16T15:07:23.061565+0000","last_became_active":"2024-09-16T15:07:22.078344+0000","last_became_peered":"2024-09-16T15:07:22.078344+0000","last_unstale":"2024-09-16T15:07:23.061565+0000","last_undegraded":"2024-09-16T15:07:23.061565+0000","last_fullsized":"2024-09-16T15:07:23.061565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:22:23.839319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774595+0000","last_change":"2024-09-16T15:07:21.737802+0000","last_active":"2024-09-16T15:07:22.774595+0000","last_peered":"2024-09-16T15:07:22.774595+0000","last_clean":"2024-09-16T15:07:22.774595+0000","last_became_active":"2024-09-16T15:07:21.737174+0000","last_became_peered":"2024-09-16T15:07:21.737174+0000","last_unstale":"2024-09-16T15:07:22.774595+0000","last_undegraded":"2024-09-16T15:07:22.774595+0000","last_fullsized":"2024-09-16T15:07:22.774595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061591+0000","last_change":"2024-09-16T15:07:22.079075+0000","last_active":"2024-09-16T15:07:23.061591+0000","last_peered":"2024-09-16T15:07:23.061591+0000","last_clean":"2024-09-16T15:07:23.061591+0000","last_became_active":"2024-09-16T15:07:22.077520+0000","last_became_peered":"2024-09-16T15:07:22.077520+0000","last_unstale":"2024-09-16T15:07:23.061591+0000","last_undegraded":"2024-09-16T15:07:23.061591+0000","last_fullsized":"2024-09-16T15:07:23.061591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:29.976832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060410+0000","last_change":"2024-09-16T15:07:21.739614+0000","last_active":"2024-09-16T15:07:23.060410+0000","last_peered":"2024-09-16T15:07:23.060410+0000","last_clean":"2024-09-16T15:07:23.060410+0000","last_became_active":"2024-09-16T15:07:21.739226+0000","last_became_peered":"2024-09-16T15:07:21.739226+0000","last_unstale":"2024-09-16T15:07:23.060410+0000","last_undegraded":"2024-09-16T15:07:23.060410+0000","last_fullsized":"2024-09-16T15:07:23.060410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:13.563410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062264+0000","last_change":"2024-09-16T15:07:22.087272+0000","last_active":"2024-09-16T15:07:23.062264+0000","last_peered":"2024-09-16T15:07:23.062264+0000","last_clean":"2024-09-16T15:07:23.062264+0000","last_became_active":"2024-09-16T15:07:22.086953+0000","last_became_peered":"2024-09-16T15:07:22.086953+0000","last_unstale":"2024-09-16T15:07:23.062264+0000","last_undegraded":"2024-09-16T15:07:23.062264+0000","last_fullsized":"2024-09-16T15:07:23.062264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:34.264644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773390+0000","last_change":"2024-09-16T15:07:21.733899+0000","last_active":"2024-09-16T15:07:22.773390+0000","last_peered":"2024-09-16T15:07:22.773390+0000","last_clean":"2024-09-16T15:07:22.773390+0000","last_became_active":"2024-09-16T15:07:21.730134+0000","last_became_peered":"2024-09-16T15:07:21.730134+0000","last_unstale":"2024-09-16T15:07:22.773390+0000","last_undegraded":"2024-09-16T15:07:22.773390+0000","last_fullsized":"2024-09-16T15:07:22.773390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.775026+0000","last_change":"2024-09-16T15:07:21.720627+0000","last_active":"2024-09-16T15:07:22.775026+0000","last_peered":"2024-09-16T15:07:22.775026+0000","last_clean":"2024-09-16T15:07:22.775026+0000","last_became_active":"2024-09-16T15:07:21.719903+0000","last_became_peered":"2024-09-16T15:07:21.719903+0000","last_unstale":"2024-09-16T15:07:22.775026+0000","last_undegraded":"2024-09-16T15:07:22.775026+0000","last_fullsized":"2024-09-16T15:07:22.775026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059146+0000","last_change":"2024-09-16T15:07:22.100423+0000","last_active":"2024-09-16T15:07:23.059146+0000","last_peered":"2024-09-16T15:07:23.059146+0000","last_clean":"2024-09-16T15:07:23.059146+0000","last_became_active":"2024-09-16T15:07:22.099384+0000","last_became_peered":"2024-09-16T15:07:22.099384+0000","last_unstale":"2024-09-16T15:07:23.059146+0000","last_undegraded":"2024-09-16T15:07:23.059146+0000","last_fullsized":"2024-09-16T15:07:23.059146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:15.595055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774917+0000","last_change":"2024-09-16T15:07:21.720105+0000","last_active":"2024-09-16T15:07:22.774917+0000","last_peered":"2024-09-16T15:07:22.774917+0000","last_clean":"2024-09-16T15:07:22.774917+0000","last_became_active":"2024-09-16T15:07:21.719254+0000","last_became_peered":"2024-09-16T15:07:21.719254+0000","last_unstale":"2024-09-16T15:07:22.774917+0000","last_undegraded":"2024-09-16T15:07:22.774917+0000","last_fullsized":"2024-09-16T15:07:22.774917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773453+0000","last_change":"2024-09-16T15:07:22.087123+0000","last_active":"2024-09-16T15:07:22.773453+0000","last_peered":"2024-09-16T15:07:22.773453+0000","last_clean":"2024-09-16T15:07:22.773453+0000","last_became_active":"2024-09-16T15:07:22.085858+0000","last_became_peered":"2024-09-16T15:07:22.085858+0000","last_unstale":"2024-09-16T15:07:22.773453+0000","last_undegraded":"2024-09-16T15:07:22.773453+0000","last_fullsized":"2024-09-16T15:07:22.773453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:43.560388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061868+0000","last_change":"2024-09-16T15:07:21.727975+0000","last_active":"2024-09-16T15:07:23.061868+0000","last_peered":"2024-09-16T15:07:23.061868+0000","last_clean":"2024-09-16T15:07:23.061868+0000","last_became_active":"2024-09-16T15:07:21.726999+0000","last_became_peered":"2024-09-16T15:07:21.726999+0000","last_unstale":"2024-09-16T15:07:23.061868+0000","last_undegraded":"2024-09-16T15:07:23.061868+0000","last_fullsized":"2024-09-16T15:07:23.061868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:26.627634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774542+0000","last_change":"2024-09-16T15:07:22.064842+0000","last_active":"2024-09-16T15:07:22.774542+0000","last_peered":"2024-09-16T15:07:22.774542+0000","last_clean":"2024-09-16T15:07:22.774542+0000","last_became_active":"2024-09-16T15:07:22.064502+0000","last_became_peered":"2024-09-16T15:07:22.064502+0000","last_unstale":"2024-09-16T15:07:22.774542+0000","last_undegraded":"2024-09-16T15:07:22.774542+0000","last_fullsized":"2024-09-16T15:07:22.774542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:51.922251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061051+0000","last_change":"2024-09-16T15:07:21.725945+0000","last_active":"2024-09-16T15:07:23.061051+0000","last_peered":"2024-09-16T15:07:23.061051+0000","last_clean":"2024-09-16T15:07:23.061051+0000","last_became_active":"2024-09-16T15:07:21.724839+0000","last_became_peered":"2024-09-16T15:07:21.724839+0000","last_unstale":"2024-09-16T15:07:23.061051+0000","last_undegraded":"2024-09-16T15:07:23.061051+0000","last_fullsized":"2024-09-16T15:07:23.061051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:25:47.932258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774660+0000","last_change":"2024-09-16T15:07:22.058944+0000","last_active":"2024-09-16T15:07:22.774660+0000","last_peered":"2024-09-16T15:07:22.774660+0000","last_clean":"2024-09-16T15:07:22.774660+0000","last_became_active":"2024-09-16T15:07:22.058700+0000","last_became_peered":"2024-09-16T15:07:22.058700+0000","last_unstale":"2024-09-16T15:07:22.774660+0000","last_undegraded":"2024-09-16T15:07:22.774660+0000","last_fullsized":"2024-09-16T15:07:22.774660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:10.707281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061840+0000","last_change":"2024-09-16T15:07:21.725144+0000","last_active":"2024-09-16T15:07:23.061840+0000","last_peered":"2024-09-16T15:07:23.061840+0000","last_clean":"2024-09-16T15:07:23.061840+0000","last_became_active":"2024-09-16T15:07:21.724855+0000","last_became_peered":"2024-09-16T15:07:21.724855+0000","last_unstale":"2024-09-16T15:07:23.061840+0000","last_undegraded":"2024-09-16T15:07:23.061840+0000","last_fullsized":"2024-09-16T15:07:23.061840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773940+0000","last_change":"2024-09-16T15:07:22.087348+0000","last_active":"2024-09-16T15:07:22.773940+0000","last_peered":"2024-09-16T15:07:22.773940+0000","last_clean":"2024-09-16T15:07:22.773940+0000","last_became_active":"2024-09-16T15:07:22.086906+0000","last_became_peered":"2024-09-16T15:07:22.086906+0000","last_unstale":"2024-09-16T15:07:22.773940+0000","last_undegraded":"2024-09-16T15:07:22.773940+0000","last_fullsized":"2024-09-16T15:07:22.773940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:53.121258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059464+0000","last_change":"2024-09-16T15:07:21.727272+0000","last_active":"2024-09-16T15:07:23.059464+0000","last_peered":"2024-09-16T15:07:23.059464+0000","last_clean":"2024-09-16T15:07:23.059464+0000","last_became_active":"2024-09-16T15:07:21.724280+0000","last_became_peered":"2024-09-16T15:07:21.724280+0000","last_unstale":"2024-09-16T15:07:23.059464+0000","last_undegraded":"2024-09-16T15:07:23.059464+0000","last_fullsized":"2024-09-16T15:07:23.059464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:50.919250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061388+0000","last_change":"2024-09-16T15:07:22.094628+0000","last_active":"2024-09-16T15:07:23.061388+0000","last_peered":"2024-09-16T15:07:23.061388+0000","last_clean":"2024-09-16T15:07:23.061388+0000","last_became_active":"2024-09-16T15:07:22.094356+0000","last_became_peered":"2024-09-16T15:07:22.094356+0000","last_unstale":"2024-09-16T15:07:23.061388+0000","last_undegraded":"2024-09-16T15:07:23.061388+0000","last_fullsized":"2024-09-16T15:07:23.061388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:37.486620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060474+0000","last_change":"2024-09-16T15:07:21.742014+0000","last_active":"2024-09-16T15:07:23.060474+0000","last_peered":"2024-09-16T15:07:23.060474+0000","last_clean":"2024-09-16T15:07:23.060474+0000","last_became_active":"2024-09-16T15:07:21.741361+0000","last_became_peered":"2024-09-16T15:07:21.741361+0000","last_unstale":"2024-09-16T15:07:23.060474+0000","last_undegraded":"2024-09-16T15:07:23.060474+0000","last_fullsized":"2024-09-16T15:07:23.060474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:05.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774833+0000","last_change":"2024-09-16T15:07:22.072156+0000","last_active":"2024-09-16T15:07:22.774833+0000","last_peered":"2024-09-16T15:07:22.774833+0000","last_clean":"2024-09-16T15:07:22.774833+0000","last_became_active":"2024-09-16T15:07:22.071767+0000","last_became_peered":"2024-09-16T15:07:22.071767+0000","last_unstale":"2024-09-16T15:07:22.774833+0000","last_undegraded":"2024-09-16T15:07:22.774833+0000","last_fullsized":"2024-09-16T15:07:22.774833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:55.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059660+0000","last_change":"2024-09-16T15:07:21.726805+0000","last_active":"2024-09-16T15:07:23.059660+0000","last_peered":"2024-09-16T15:07:23.059660+0000","last_clean":"2024-09-16T15:07:23.059660+0000","last_became_active":"2024-09-16T15:07:21.726154+0000","last_became_peered":"2024-09-16T15:07:21.726154+0000","last_unstale":"2024-09-16T15:07:23.059660+0000","last_undegraded":"2024-09-16T15:07:23.059660+0000","last_fullsized":"2024-09-16T15:07:23.059660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:21.077487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774116+0000","last_change":"2024-09-16T15:07:22.088691+0000","last_active":"2024-09-16T15:07:22.774116+0000","last_peered":"2024-09-16T15:07:22.774116+0000","last_clean":"2024-09-16T15:07:22.774116+0000","last_became_active":"2024-09-16T15:07:22.087008+0000","last_became_peered":"2024-09-16T15:07:22.087008+0000","last_unstale":"2024-09-16T15:07:22.774116+0000","last_undegraded":"2024-09-16T15:07:22.774116+0000","last_fullsized":"2024-09-16T15:07:22.774116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:07.641636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":167,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.091073+0000","last_change":"2024-09-16T15:07:11.063383+0000","last_active":"2024-09-16T15:07:22.091073+0000","last_peered":"2024-09-16T15:07:22.091073+0000","last_clean":"2024-09-16T15:07:22.091073+0000","last_became_active":"2024-09-16T15:07:11.062607+0000","last_became_peered":"2024-09-16T15:07:11.062607+0000","last_unstale":"2024-09-16T15:07:22.091073+0000","last_undegraded":"2024-09-16T15:07:22.091073+0000","last_fullsized":"2024-09-16T15:07:22.091073+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:09.570910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:09.570910+0000","last_clean_scrub_stamp":"2024-09-16T15:07:09.570910+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:23.466827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.237221+0000","last_change":"2024-09-16T15:07:17.669610+0000","last_active":"2024-09-16T15:07:27.237221+0000","last_peered":"2024-09-16T15:07:27.237221+0000","last_clean":"2024-09-16T15:07:27.237221+0000","last_became_active":"2024-09-16T15:07:15.657618+0000","last_became_peered":"2024-09-16T15:07:15.657618+0000","last_unstale":"2024-09-16T15:07:27.237221+0000","last_undegraded":"2024-09-16T15:07:27.237221+0000","last_fullsized":"2024-09-16T15:07:27.237221+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:28:19.381043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085582900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059886+0000","last_change":"2024-09-16T15:07:21.741970+0000","last_active":"2024-09-16T15:07:23.059886+0000","last_peered":"2024-09-16T15:07:23.059886+0000","last_clean":"2024-09-16T15:07:23.059886+0000","last_became_active":"2024-09-16T15:07:21.741305+0000","last_became_peered":"2024-09-16T15:07:21.741305+0000","last_unstale":"2024-09-16T15:07:23.059886+0000","last_undegraded":"2024-09-16T15:07:23.059886+0000","last_fullsized":"2024-09-16T15:07:23.059886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:46.566147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061679+0000","last_change":"2024-09-16T15:07:22.095031+0000","last_active":"2024-09-16T15:07:23.061679+0000","last_peered":"2024-09-16T15:07:23.061679+0000","last_clean":"2024-09-16T15:07:23.061679+0000","last_became_active":"2024-09-16T15:07:22.094793+0000","last_became_peered":"2024-09-16T15:07:22.094793+0000","last_unstale":"2024-09-16T15:07:23.061679+0000","last_undegraded":"2024-09-16T15:07:23.061679+0000","last_fullsized":"2024-09-16T15:07:23.061679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:38:36.377299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.235522+0000","last_change":"2024-09-16T15:07:17.669433+0000","last_active":"2024-09-16T15:07:27.235522+0000","last_peered":"2024-09-16T15:07:27.235522+0000","last_clean":"2024-09-16T15:07:27.235522+0000","last_became_active":"2024-09-16T15:07:15.656237+0000","last_became_peered":"2024-09-16T15:07:15.656237+0000","last_unstale":"2024-09-16T15:07:27.235522+0000","last_undegraded":"2024-09-16T15:07:27.235522+0000","last_fullsized":"2024-09-16T15:07:27.235522+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:49.124474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068265100000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061086+0000","last_change":"2024-09-16T15:07:21.713506+0000","last_active":"2024-09-16T15:07:23.061086+0000","last_peered":"2024-09-16T15:07:23.061086+0000","last_clean":"2024-09-16T15:07:23.061086+0000","last_became_active":"2024-09-16T15:07:21.713226+0000","last_became_peered":"2024-09-16T15:07:21.713226+0000","last_unstale":"2024-09-16T15:07:23.061086+0000","last_undegraded":"2024-09-16T15:07:23.061086+0000","last_fullsized":"2024-09-16T15:07:23.061086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061755+0000","last_change":"2024-09-16T15:07:22.079061+0000","last_active":"2024-09-16T15:07:23.061755+0000","last_peered":"2024-09-16T15:07:23.061755+0000","last_clean":"2024-09-16T15:07:23.061755+0000","last_became_active":"2024-09-16T15:07:22.076697+0000","last_became_peered":"2024-09-16T15:07:22.076697+0000","last_unstale":"2024-09-16T15:07:23.061755+0000","last_undegraded":"2024-09-16T15:07:23.061755+0000","last_fullsized":"2024-09-16T15:07:23.061755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:58.996879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.054623+0000","last_change":"2024-09-16T15:07:18.173036+0000","last_active":"2024-09-16T15:07:22.054623+0000","last_peered":"2024-09-16T15:07:22.054623+0000","last_clean":"2024-09-16T15:07:22.054623+0000","last_became_active":"2024-09-16T15:07:15.656238+0000","last_became_peered":"2024-09-16T15:07:15.656238+0000","last_unstale":"2024-09-16T15:07:22.054623+0000","last_undegraded":"2024-09-16T15:07:22.054623+0000","last_fullsized":"2024-09-16T15:07:22.054623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:15:02.977857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080979399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060974+0000","last_change":"2024-09-16T15:07:21.722652+0000","last_active":"2024-09-16T15:07:23.060974+0000","last_peered":"2024-09-16T15:07:23.060974+0000","last_clean":"2024-09-16T15:07:23.060974+0000","last_became_active":"2024-09-16T15:07:21.722308+0000","last_became_peered":"2024-09-16T15:07:21.722308+0000","last_unstale":"2024-09-16T15:07:23.060974+0000","last_undegraded":"2024-09-16T15:07:23.060974+0000","last_fullsized":"2024-09-16T15:07:23.060974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:40.604572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062351+0000","last_change":"2024-09-16T15:07:22.087268+0000","last_active":"2024-09-16T15:07:23.062351+0000","last_peered":"2024-09-16T15:07:23.062351+0000","last_clean":"2024-09-16T15:07:23.062351+0000","last_became_active":"2024-09-16T15:07:22.086932+0000","last_became_peered":"2024-09-16T15:07:22.086932+0000","last_unstale":"2024-09-16T15:07:23.062351+0000","last_undegraded":"2024-09-16T15:07:23.062351+0000","last_fullsized":"2024-09-16T15:07:23.062351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:55.204636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.236329+0000","last_change":"2024-09-16T15:07:17.671010+0000","last_active":"2024-09-16T15:07:27.236329+0000","last_peered":"2024-09-16T15:07:27.236329+0000","last_clean":"2024-09-16T15:07:27.236329+0000","last_became_active":"2024-09-16T15:07:15.658625+0000","last_became_peered":"2024-09-16T15:07:15.658625+0000","last_unstale":"2024-09-16T15:07:27.236329+0000","last_undegraded":"2024-09-16T15:07:27.236329+0000","last_fullsized":"2024-09-16T15:07:27.236329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:18.695305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00224435,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774699+0000","last_change":"2024-09-16T15:07:21.712013+0000","last_active":"2024-09-16T15:07:22.774699+0000","last_peered":"2024-09-16T15:07:22.774699+0000","last_clean":"2024-09-16T15:07:22.774699+0000","last_became_active":"2024-09-16T15:07:21.710551+0000","last_became_peered":"2024-09-16T15:07:21.710551+0000","last_unstale":"2024-09-16T15:07:22.774699+0000","last_undegraded":"2024-09-16T15:07:22.774699+0000","last_fullsized":"2024-09-16T15:07:22.774699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061756+0000","last_change":"2024-09-16T15:07:22.067593+0000","last_active":"2024-09-16T15:07:23.061756+0000","last_peered":"2024-09-16T15:07:23.061756+0000","last_clean":"2024-09-16T15:07:23.061756+0000","last_became_active":"2024-09-16T15:07:22.067254+0000","last_became_peered":"2024-09-16T15:07:22.067254+0000","last_unstale":"2024-09-16T15:07:23.061756+0000","last_undegraded":"2024-09-16T15:07:23.061756+0000","last_fullsized":"2024-09-16T15:07:23.061756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:08:43.560388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774774+0000","last_change":"2024-09-16T15:07:21.711346+0000","last_active":"2024-09-16T15:07:22.774774+0000","last_peered":"2024-09-16T15:07:22.774774+0000","last_clean":"2024-09-16T15:07:22.774774+0000","last_became_active":"2024-09-16T15:07:21.710693+0000","last_became_peered":"2024-09-16T15:07:21.710693+0000","last_unstale":"2024-09-16T15:07:22.774774+0000","last_undegraded":"2024-09-16T15:07:22.774774+0000","last_fullsized":"2024-09-16T15:07:22.774774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:50.814161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.234416+0000","last_change":"2024-09-16T15:07:17.664832+0000","last_active":"2024-09-16T15:07:27.234416+0000","last_peered":"2024-09-16T15:07:27.234416+0000","last_clean":"2024-09-16T15:07:27.234416+0000","last_became_active":"2024-09-16T15:07:15.654955+0000","last_became_peered":"2024-09-16T15:07:15.654955+0000","last_unstale":"2024-09-16T15:07:27.234416+0000","last_undegraded":"2024-09-16T15:07:27.234416+0000","last_fullsized":"2024-09-16T15:07:27.234416+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:48:38.399733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085315100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773630+0000","last_change":"2024-09-16T15:07:22.091087+0000","last_active":"2024-09-16T15:07:22.773630+0000","last_peered":"2024-09-16T15:07:22.773630+0000","last_clean":"2024-09-16T15:07:22.773630+0000","last_became_active":"2024-09-16T15:07:22.088585+0000","last_became_peered":"2024-09-16T15:07:22.088585+0000","last_unstale":"2024-09-16T15:07:22.773630+0000","last_undegraded":"2024-09-16T15:07:22.773630+0000","last_fullsized":"2024-09-16T15:07:22.773630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:19.377720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061723+0000","last_change":"2024-09-16T15:07:21.713440+0000","last_active":"2024-09-16T15:07:23.061723+0000","last_peered":"2024-09-16T15:07:23.061723+0000","last_clean":"2024-09-16T15:07:23.061723+0000","last_became_active":"2024-09-16T15:07:21.713127+0000","last_became_peered":"2024-09-16T15:07:21.713127+0000","last_unstale":"2024-09-16T15:07:23.061723+0000","last_undegraded":"2024-09-16T15:07:23.061723+0000","last_fullsized":"2024-09-16T15:07:23.061723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.239668+0000","last_change":"2024-09-16T15:07:17.671471+0000","last_active":"2024-09-16T15:07:27.239668+0000","last_peered":"2024-09-16T15:07:27.239668+0000","last_clean":"2024-09-16T15:07:27.239668+0000","last_became_active":"2024-09-16T15:07:15.656288+0000","last_became_peered":"2024-09-16T15:07:15.656288+0000","last_unstale":"2024-09-16T15:07:27.239668+0000","last_undegraded":"2024-09-16T15:07:27.239668+0000","last_fullsized":"2024-09-16T15:07:27.239668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:03.700310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026747759999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773852+0000","last_change":"2024-09-16T15:07:22.087230+0000","last_active":"2024-09-16T15:07:22.773852+0000","last_peered":"2024-09-16T15:07:22.773852+0000","last_clean":"2024-09-16T15:07:22.773852+0000","last_became_active":"2024-09-16T15:07:22.086917+0000","last_became_peered":"2024-09-16T15:07:22.086917+0000","last_unstale":"2024-09-16T15:07:22.773852+0000","last_undegraded":"2024-09-16T15:07:22.773852+0000","last_fullsized":"2024-09-16T15:07:22.773852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:05.650468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061782+0000","last_change":"2024-09-16T15:07:21.727506+0000","last_active":"2024-09-16T15:07:23.061782+0000","last_peered":"2024-09-16T15:07:23.061782+0000","last_clean":"2024-09-16T15:07:23.061782+0000","last_became_active":"2024-09-16T15:07:21.724965+0000","last_became_peered":"2024-09-16T15:07:21.724965+0000","last_unstale":"2024-09-16T15:07:23.061782+0000","last_undegraded":"2024-09-16T15:07:23.061782+0000","last_fullsized":"2024-09-16T15:07:23.061782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:41.835469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.234823+0000","last_change":"2024-09-16T15:07:17.664973+0000","last_active":"2024-09-16T15:07:27.234823+0000","last_peered":"2024-09-16T15:07:27.234823+0000","last_clean":"2024-09-16T15:07:27.234823+0000","last_became_active":"2024-09-16T15:07:15.655047+0000","last_became_peered":"2024-09-16T15:07:15.655047+0000","last_unstale":"2024-09-16T15:07:27.234823+0000","last_undegraded":"2024-09-16T15:07:27.234823+0000","last_fullsized":"2024-09-16T15:07:27.234823+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:05:53.308490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081824399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059410+0000","last_change":"2024-09-16T15:07:22.100811+0000","last_active":"2024-09-16T15:07:23.059410+0000","last_peered":"2024-09-16T15:07:23.059410+0000","last_clean":"2024-09-16T15:07:23.059410+0000","last_became_active":"2024-09-16T15:07:22.099732+0000","last_became_peered":"2024-09-16T15:07:22.099732+0000","last_unstale":"2024-09-16T15:07:23.059410+0000","last_undegraded":"2024-09-16T15:07:23.059410+0000","last_fullsized":"2024-09-16T15:07:23.059410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:36.011335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.057562+0000","last_change":"2024-09-16T15:07:21.733464+0000","last_active":"2024-09-16T15:07:23.057562+0000","last_peered":"2024-09-16T15:07:23.057562+0000","last_clean":"2024-09-16T15:07:23.057562+0000","last_became_active":"2024-09-16T15:07:21.732659+0000","last_became_peered":"2024-09-16T15:07:21.732659+0000","last_unstale":"2024-09-16T15:07:23.057562+0000","last_undegraded":"2024-09-16T15:07:23.057562+0000","last_fullsized":"2024-09-16T15:07:23.057562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:42:34.995957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:07:27.238873+0000","last_change":"2024-09-16T15:07:17.671161+0000","last_active":"2024-09-16T15:07:27.238873+0000","last_peered":"2024-09-16T15:07:27.238873+0000","last_clean":"2024-09-16T15:07:27.238873+0000","last_became_active":"2024-09-16T15:07:15.654237+0000","last_became_peered":"2024-09-16T15:07:15.654237+0000","last_unstale":"2024-09-16T15:07:27.238873+0000","last_undegraded":"2024-09-16T15:07:27.238873+0000","last_fullsized":"2024-09-16T15:07:27.238873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:14.613114+0000","last_clean_scrub_stamp":"2024-09-16T15:07:14.613114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:56.780961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002356239,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774462+0000","last_change":"2024-09-16T15:07:22.071750+0000","last_active":"2024-09-16T15:07:22.774462+0000","last_peered":"2024-09-16T15:07:22.774462+0000","last_clean":"2024-09-16T15:07:22.774462+0000","last_became_active":"2024-09-16T15:07:22.071469+0000","last_became_peered":"2024-09-16T15:07:22.071469+0000","last_unstale":"2024-09-16T15:07:22.774462+0000","last_undegraded":"2024-09-16T15:07:22.774462+0000","last_fullsized":"2024-09-16T15:07:22.774462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:45.906789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774639+0000","last_change":"2024-09-16T15:07:22.080700+0000","last_active":"2024-09-16T15:07:22.774639+0000","last_peered":"2024-09-16T15:07:22.774639+0000","last_clean":"2024-09-16T15:07:22.774639+0000","last_became_active":"2024-09-16T15:07:22.080300+0000","last_became_peered":"2024-09-16T15:07:22.080300+0000","last_unstale":"2024-09-16T15:07:22.774639+0000","last_undegraded":"2024-09-16T15:07:22.774639+0000","last_fullsized":"2024-09-16T15:07:22.774639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:27.703305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060850+0000","last_change":"2024-09-16T15:07:21.741959+0000","last_active":"2024-09-16T15:07:23.060850+0000","last_peered":"2024-09-16T15:07:23.060850+0000","last_clean":"2024-09-16T15:07:23.060850+0000","last_became_active":"2024-09-16T15:07:21.741329+0000","last_became_peered":"2024-09-16T15:07:21.741329+0000","last_unstale":"2024-09-16T15:07:23.060850+0000","last_undegraded":"2024-09-16T15:07:23.060850+0000","last_fullsized":"2024-09-16T15:07:23.060850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:31.395563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773674+0000","last_change":"2024-09-16T15:07:22.088939+0000","last_active":"2024-09-16T15:07:22.773674+0000","last_peered":"2024-09-16T15:07:22.773674+0000","last_clean":"2024-09-16T15:07:22.773674+0000","last_became_active":"2024-09-16T15:07:22.087096+0000","last_became_peered":"2024-09-16T15:07:22.087096+0000","last_unstale":"2024-09-16T15:07:22.773674+0000","last_undegraded":"2024-09-16T15:07:22.773674+0000","last_fullsized":"2024-09-16T15:07:22.773674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:49.184359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774748+0000","last_change":"2024-09-16T15:07:21.737806+0000","last_active":"2024-09-16T15:07:22.774748+0000","last_peered":"2024-09-16T15:07:22.774748+0000","last_clean":"2024-09-16T15:07:22.774748+0000","last_became_active":"2024-09-16T15:07:21.737289+0000","last_became_peered":"2024-09-16T15:07:21.737289+0000","last_unstale":"2024-09-16T15:07:22.774748+0000","last_undegraded":"2024-09-16T15:07:22.774748+0000","last_fullsized":"2024-09-16T15:07:22.774748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:15.691812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774875+0000","last_change":"2024-09-16T15:07:21.738040+0000","last_active":"2024-09-16T15:07:22.774875+0000","last_peered":"2024-09-16T15:07:22.774875+0000","last_clean":"2024-09-16T15:07:22.774875+0000","last_became_active":"2024-09-16T15:07:21.737276+0000","last_became_peered":"2024-09-16T15:07:21.737276+0000","last_unstale":"2024-09-16T15:07:22.774875+0000","last_undegraded":"2024-09-16T15:07:22.774875+0000","last_fullsized":"2024-09-16T15:07:22.774875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061179+0000","last_change":"2024-09-16T15:07:22.073415+0000","last_active":"2024-09-16T15:07:23.061179+0000","last_peered":"2024-09-16T15:07:23.061179+0000","last_clean":"2024-09-16T15:07:23.061179+0000","last_became_active":"2024-09-16T15:07:22.073160+0000","last_became_peered":"2024-09-16T15:07:22.073160+0000","last_unstale":"2024-09-16T15:07:23.061179+0000","last_undegraded":"2024-09-16T15:07:23.061179+0000","last_fullsized":"2024-09-16T15:07:23.061179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:32.398564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774167+0000","last_change":"2024-09-16T15:07:21.730081+0000","last_active":"2024-09-16T15:07:22.774167+0000","last_peered":"2024-09-16T15:07:22.774167+0000","last_clean":"2024-09-16T15:07:22.774167+0000","last_became_active":"2024-09-16T15:07:21.729450+0000","last_became_peered":"2024-09-16T15:07:21.729450+0000","last_unstale":"2024-09-16T15:07:22.774167+0000","last_undegraded":"2024-09-16T15:07:22.774167+0000","last_fullsized":"2024-09-16T15:07:22.774167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:49.290679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.058880+0000","last_change":"2024-09-16T15:07:22.100315+0000","last_active":"2024-09-16T15:07:23.058880+0000","last_peered":"2024-09-16T15:07:23.058880+0000","last_clean":"2024-09-16T15:07:23.058880+0000","last_became_active":"2024-09-16T15:07:22.099367+0000","last_became_peered":"2024-09-16T15:07:22.099367+0000","last_unstale":"2024-09-16T15:07:23.058880+0000","last_undegraded":"2024-09-16T15:07:23.058880+0000","last_fullsized":"2024-09-16T15:07:23.058880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:45.128492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061737+0000","last_change":"2024-09-16T15:07:22.079005+0000","last_active":"2024-09-16T15:07:23.061737+0000","last_peered":"2024-09-16T15:07:23.061737+0000","last_clean":"2024-09-16T15:07:23.061737+0000","last_became_active":"2024-09-16T15:07:22.077888+0000","last_became_peered":"2024-09-16T15:07:22.077888+0000","last_unstale":"2024-09-16T15:07:23.061737+0000","last_undegraded":"2024-09-16T15:07:23.061737+0000","last_fullsized":"2024-09-16T15:07:23.061737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:21.103662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.059542+0000","last_change":"2024-09-16T15:07:21.724826+0000","last_active":"2024-09-16T15:07:23.059542+0000","last_peered":"2024-09-16T15:07:23.059542+0000","last_clean":"2024-09-16T15:07:23.059542+0000","last_became_active":"2024-09-16T15:07:21.724152+0000","last_became_peered":"2024-09-16T15:07:21.724152+0000","last_unstale":"2024-09-16T15:07:23.059542+0000","last_undegraded":"2024-09-16T15:07:23.059542+0000","last_fullsized":"2024-09-16T15:07:23.059542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:26.700305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062278+0000","last_change":"2024-09-16T15:07:22.087018+0000","last_active":"2024-09-16T15:07:23.062278+0000","last_peered":"2024-09-16T15:07:23.062278+0000","last_clean":"2024-09-16T15:07:23.062278+0000","last_became_active":"2024-09-16T15:07:22.086746+0000","last_became_peered":"2024-09-16T15:07:22.086746+0000","last_unstale":"2024-09-16T15:07:23.062278+0000","last_undegraded":"2024-09-16T15:07:23.062278+0000","last_fullsized":"2024-09-16T15:07:23.062278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:11.167685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773475+0000","last_change":"2024-09-16T15:07:21.726511+0000","last_active":"2024-09-16T15:07:22.773475+0000","last_peered":"2024-09-16T15:07:22.773475+0000","last_clean":"2024-09-16T15:07:22.773475+0000","last_became_active":"2024-09-16T15:07:21.725971+0000","last_became_peered":"2024-09-16T15:07:21.725971+0000","last_unstale":"2024-09-16T15:07:22.773475+0000","last_undegraded":"2024-09-16T15:07:22.773475+0000","last_fullsized":"2024-09-16T15:07:22.773475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:52:15.691812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061171+0000","last_change":"2024-09-16T15:07:22.093131+0000","last_active":"2024-09-16T15:07:23.061171+0000","last_peered":"2024-09-16T15:07:23.061171+0000","last_clean":"2024-09-16T15:07:23.061171+0000","last_became_active":"2024-09-16T15:07:22.092810+0000","last_became_peered":"2024-09-16T15:07:22.092810+0000","last_unstale":"2024-09-16T15:07:23.061171+0000","last_undegraded":"2024-09-16T15:07:23.061171+0000","last_fullsized":"2024-09-16T15:07:23.061171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:27.872651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773764+0000","last_change":"2024-09-16T15:07:21.709700+0000","last_active":"2024-09-16T15:07:22.773764+0000","last_peered":"2024-09-16T15:07:22.773764+0000","last_clean":"2024-09-16T15:07:22.773764+0000","last_became_active":"2024-09-16T15:07:21.708187+0000","last_became_peered":"2024-09-16T15:07:21.708187+0000","last_unstale":"2024-09-16T15:07:22.773764+0000","last_undegraded":"2024-09-16T15:07:22.773764+0000","last_fullsized":"2024-09-16T15:07:22.773764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:07:50.814161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061551+0000","last_change":"2024-09-16T15:07:22.067775+0000","last_active":"2024-09-16T15:07:23.061551+0000","last_peered":"2024-09-16T15:07:23.061551+0000","last_clean":"2024-09-16T15:07:23.061551+0000","last_became_active":"2024-09-16T15:07:22.067259+0000","last_became_peered":"2024-09-16T15:07:22.067259+0000","last_unstale":"2024-09-16T15:07:23.061551+0000","last_undegraded":"2024-09-16T15:07:23.061551+0000","last_fullsized":"2024-09-16T15:07:23.061551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:04.412982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060444+0000","last_change":"2024-09-16T15:07:21.726917+0000","last_active":"2024-09-16T15:07:23.060444+0000","last_peered":"2024-09-16T15:07:23.060444+0000","last_clean":"2024-09-16T15:07:23.060444+0000","last_became_active":"2024-09-16T15:07:21.724806+0000","last_became_peered":"2024-09-16T15:07:21.724806+0000","last_unstale":"2024-09-16T15:07:23.060444+0000","last_undegraded":"2024-09-16T15:07:23.060444+0000","last_fullsized":"2024-09-16T15:07:23.060444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:54.201635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774618+0000","last_change":"2024-09-16T15:07:22.059433+0000","last_active":"2024-09-16T15:07:22.774618+0000","last_peered":"2024-09-16T15:07:22.774618+0000","last_clean":"2024-09-16T15:07:22.774618+0000","last_became_active":"2024-09-16T15:07:22.058973+0000","last_became_peered":"2024-09-16T15:07:22.058973+0000","last_unstale":"2024-09-16T15:07:22.774618+0000","last_undegraded":"2024-09-16T15:07:22.774618+0000","last_fullsized":"2024-09-16T15:07:22.774618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:11.167685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.773963+0000","last_change":"2024-09-16T15:07:21.726761+0000","last_active":"2024-09-16T15:07:22.773963+0000","last_peered":"2024-09-16T15:07:22.773963+0000","last_clean":"2024-09-16T15:07:22.773963+0000","last_became_active":"2024-09-16T15:07:21.726186+0000","last_became_peered":"2024-09-16T15:07:21.726186+0000","last_unstale":"2024-09-16T15:07:22.773963+0000","last_undegraded":"2024-09-16T15:07:22.773963+0000","last_fullsized":"2024-09-16T15:07:22.773963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:55.604228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774571+0000","last_change":"2024-09-16T15:07:22.059781+0000","last_active":"2024-09-16T15:07:22.774571+0000","last_peered":"2024-09-16T15:07:22.774571+0000","last_clean":"2024-09-16T15:07:22.774571+0000","last_became_active":"2024-09-16T15:07:22.059423+0000","last_became_peered":"2024-09-16T15:07:22.059423+0000","last_unstale":"2024-09-16T15:07:22.774571+0000","last_undegraded":"2024-09-16T15:07:22.774571+0000","last_fullsized":"2024-09-16T15:07:22.774571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:22.080488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.061523+0000","last_change":"2024-09-16T15:07:21.743147+0000","last_active":"2024-09-16T15:07:23.061523+0000","last_peered":"2024-09-16T15:07:23.061523+0000","last_clean":"2024-09-16T15:07:23.061523+0000","last_became_active":"2024-09-16T15:07:21.742365+0000","last_became_peered":"2024-09-16T15:07:21.742365+0000","last_unstale":"2024-09-16T15:07:23.061523+0000","last_undegraded":"2024-09-16T15:07:23.061523+0000","last_fullsized":"2024-09-16T15:07:23.061523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:10.164684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.060096+0000","last_change":"2024-09-16T15:07:22.076343+0000","last_active":"2024-09-16T15:07:23.060096+0000","last_peered":"2024-09-16T15:07:23.060096+0000","last_clean":"2024-09-16T15:07:23.060096+0000","last_became_active":"2024-09-16T15:07:22.075982+0000","last_became_peered":"2024-09-16T15:07:22.075982+0000","last_unstale":"2024-09-16T15:07:23.060096+0000","last_undegraded":"2024-09-16T15:07:23.060096+0000","last_fullsized":"2024-09-16T15:07:23.060096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:25:32.065204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-16T15:07:23.062410+0000","last_change":"2024-09-16T15:07:21.722589+0000","last_active":"2024-09-16T15:07:23.062410+0000","last_peered":"2024-09-16T15:07:23.062410+0000","last_clean":"2024-09-16T15:07:23.062410+0000","last_became_active":"2024-09-16T15:07:21.722226+0000","last_became_peered":"2024-09-16T15:07:21.722226+0000","last_unstale":"2024-09-16T15:07:23.062410+0000","last_undegraded":"2024-09-16T15:07:23.062410+0000","last_fullsized":"2024-09-16T15:07:23.062410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774405+0000","last_change":"2024-09-16T15:07:22.061731+0000","last_active":"2024-09-16T15:07:22.774405+0000","last_peered":"2024-09-16T15:07:22.774405+0000","last_clean":"2024-09-16T15:07:22.774405+0000","last_became_active":"2024-09-16T15:07:22.061419+0000","last_became_peered":"2024-09-16T15:07:22.061419+0000","last_unstale":"2024-09-16T15:07:22.774405+0000","last_undegraded":"2024-09-16T15:07:22.774405+0000","last_fullsized":"2024-09-16T15:07:22.774405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:21.668823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:21.668823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:47.569148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-16T15:07:22.774142+0000","last_change":"2024-09-16T15:07:21.709152+0000","last_active":"2024-09-16T15:07:22.774142+0000","last_peered":"2024-09-16T15:07:22.774142+0000","last_clean":"2024-09-16T15:07:22.774142+0000","last_became_active":"2024-09-16T15:07:21.708468+0000","last_became_peered":"2024-09-16T15:07:21.708468+0000","last_unstale":"2024-09-16T15:07:22.774142+0000","last_undegraded":"2024-09-16T15:07:22.774142+0000","last_fullsized":"2024-09-16T15:07:22.774142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:07:20.665823+0000","last_clean_scrub_stamp":"2024-09-16T15:07:20.665823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:57.664958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705671,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":69541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":2,"up_from":9,"seq":38654705672,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":659842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]}]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":658806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":68365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:07:29.806 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:07:29.806 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T15:07:29.806 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T15:07:29.806 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T15:07:29.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ec5fb9640 1 Processor -- start 2024-09-16T15:07:29.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ec5fb9640 1 -- start start 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ec5fb9640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0ec00a5960 0x7f0ec009e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ec5fb9640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec006beb0 con 0x7f0ec00a28c0 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ec5fb9640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec00a62e0 con 0x7f0ec00a1dd0 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ec5fb9640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec006b180 con 0x7f0ec00a5960 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0ec00a28c0 0x7f0ec014a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34010/0 (socket says 172.21.15.133:34010) 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebffff640 1 -- 172.21.15.133:0/1354296011 learned_addr learned my addr 172.21.15.133:0/1354296011 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebf7fe640 1 --2- 172.21.15.133:0/1354296011 >> v2:172.21.15.191:3300/0 conn(0x7f0ec00a5960 0x7f0ec009e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105651049 0 0) 0x7f0ec006beb0 con 0x7f0ec00a28c0 2024-09-16T15:07:29.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea8003530 con 0x7f0ec00a28c0 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2668408968 0 0) 0x7f0ea8003530 con 0x7f0ec00a28c0 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 >> v2:172.21.15.191:3300/0 conn(0x7f0ec00a5960 msgr2=0x7f0ec009e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 --2- 172.21.15.133:0/1354296011 >> v2:172.21.15.191:3300/0 conn(0x7f0ec00a5960 0x7f0ec009e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 >> v1:172.21.15.191:6789/0 conn(0x7f0ec00a1dd0 legacy=0x7f0ec00a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec006b420 con 0x7f0ec00a28c0 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f0eb8001a00 con 0x7f0ec00a28c0 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 >> v1:172.21.15.133:6789/0 conn(0x7f0ec00a28c0 legacy=0x7f0ec014a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ea8003980 0x7f0ea8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ea8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea80052b0 0x7f0ea800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec006b180 con 0x7f0ea80047d0 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea800c180 con 0x7f0ea8003980 2024-09-16T15:07:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea800c340 con 0x7f0ea80052b0 2024-09-16T15:07:29.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebeffd640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ea8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebffff640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea80052b0 0x7f0ea800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.962+0000 7f0ebf7fe640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ea8003980 0x7f0ea8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ebeffd640 1 -- 172.21.15.133:0/1354296011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea80052b0 msgr2=0x7f0ea800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ebeffd640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea80052b0 0x7f0ea800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ebeffd640 1 -- 172.21.15.133:0/1354296011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ea8003980 msgr2=0x7f0ea8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:29.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ebeffd640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ea8003980 0x7f0ea8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ebeffd640 1 -- 172.21.15.133:0/1354296011 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea8003530 con 0x7f0ea80047d0 2024-09-16T15:07:29.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ebffff640 1 -- 172.21.15.133:0/1354296011 reap_dead start 2024-09-16T15:07:29.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ebeffd640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ea8004c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40017d0 tx=0x7f0eb401ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459d9ee4c7e0298e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40052d0 con 0x7f0ea80047d0 2024-09-16T15:07:29.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb4005430 con 0x7f0ea80047d0 2024-09-16T15:07:29.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ebe7fc640 1 -- 172.21.15.133:0/1354296011 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40a7970 con 0x7f0ea80047d0 2024-09-16T15:07:29.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/1354296011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 msgr2=0x7f0ea8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ec5fb9640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ea8004c50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40017d0 tx=0x7f0eb401ea50 comp rx=0 tx=0).stop 2024-09-16T15:07:29.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/1354296011 shutdown_connections 2024-09-16T15:07:29.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ec5fb9640 1 --2- 172.21.15.133:0/1354296011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ea8004c50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/1354296011 >> 172.21.15.133:0/1354296011 conn(0x7f0ec00a3fd0 msgr2=0x7f0ec00a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:29.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/1354296011 shutdown_connections 2024-09-16T15:07:29.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.966+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/1354296011 wait complete. 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 Processor -- start 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 -- start start 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ec009d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ec00ab850 0x7f0ec00abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0070340 0x7f0ec00707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00aae60 con 0x7f0ea80047d0 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00b22e0 con 0x7f0ec0070340 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01dbf20 con 0x7f0ec00ab850 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ec009d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebeffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ec00ab850 0x7f0ec00abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebf7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ec009d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57476/0 (socket says 172.21.15.133:57476) 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebf7fe640 1 -- 172.21.15.133:0/3113563550 learned_addr learned my addr 172.21.15.133:0/3113563550 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebffff640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0070340 0x7f0ec00707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebeffd640 1 -- 172.21.15.133:0/3113563550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0070340 msgr2=0x7f0ec00707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebeffd640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0070340 0x7f0ec00707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebeffd640 1 -- 172.21.15.133:0/3113563550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 msgr2=0x7f0ec009d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebeffd640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ec009d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebeffd640 1 -- 172.21.15.133:0/3113563550 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec01dc1e0 con 0x7f0ec00ab850 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebf7fe640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ec009d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ebeffd640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ec00ab850 0x7f0ec00abc50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4004000 tx=0x7f0ea8005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ea7fff640 1 -- 172.21.15.133:0/3113563550 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4007a40 con 0x7f0ec00ab850 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ea7fff640 1 -- 172.21.15.133:0/3113563550 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb40a7e10 con 0x7f0ec00ab850 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/3113563550 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec00ab090 con 0x7f0ec00ab850 2024-09-16T15:07:29.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ea7fff640 1 -- 172.21.15.133:0/3113563550 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40052d0 con 0x7f0ec00ab850 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/3113563550 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec0071530 con 0x7f0ec00ab850 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/3113563550 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec00a53d0 con 0x7f0ec00ab850 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ea7fff640 1 -- 172.21.15.133:0/3113563550 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f0eb4007030 con 0x7f0ec00ab850 2024-09-16T15:07:29.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.970+0000 7f0ea7fff640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.974+0000 7f0ea7fff640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.974+0000 7f0ea7fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.974+0000 7f0ea7fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.974+0000 7f0ea7fff640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0eac097840 0x7f0eac099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.974+0000 7f0ea7fff640 1 -- 172.21.15.133:0/3113563550 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f0eb4159840 con 0x7f0ec00ab850 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.974+0000 7f0ebf7fe640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0eac097840 0x7f0eac099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:29.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.974+0000 7f0ebf7fe640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0eac097840 0x7f0eac099d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00145c0 tx=0x7f0eb000a2a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:29.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:29.978+0000 7f0ea7fff640 1 -- 172.21.15.133:0/3113563550 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0eb4126f50 con 0x7f0ec00ab850 2024-09-16T15:07:30.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.154+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/3113563550 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0ec00a53d0 con 0x7f0ec00ab850 2024-09-16T15:07:30.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.154+0000 7f0ea7fff640 1 -- 172.21.15.133:0/3113563550 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0eb4128e80 con 0x7f0ec00ab850 2024-09-16T15:07:30.160 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:30.160 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T15:07:30.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 10 mgrc shutdown 2024-09-16T15:07:30.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/3113563550 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0eac097840 msgr2=0x7f0eac099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0eac097840 0x7f0eac099d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00145c0 tx=0x7f0eb000a2a0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/3113563550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ec00ab850 msgr2=0x7f0ec00abc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ec00ab850 0x7f0ec00abc50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4004000 tx=0x7f0ea8005800 comp rx=0 tx=0).stop 2024-09-16T15:07:30.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/3113563550 shutdown_connections 2024-09-16T15:07:30.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0070340 0x7f0ec00707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ec00ab850 0x7f0ec00abc50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f0eac097840 0x7f0eac099d00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 --2- 172.21.15.133:0/3113563550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea80047d0 0x7f0ec009d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/3113563550 >> 172.21.15.133:0/3113563550 conn(0x7f0ec00a3fd0 msgr2=0x7f0ec00b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:30.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/3113563550 shutdown_connections 2024-09-16T15:07:30.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.158+0000 7f0ec5fb9640 1 -- 172.21.15.133:0/3113563550 wait complete. 2024-09-16T15:07:30.178 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T15:07:30.178 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:30.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f808200d640 1 Processor -- start 2024-09-16T15:07:30.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f808200d640 1 -- start start 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f808200d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c06f400 0x7f807c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f808200d640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c066340 con 0x7f807c06c030 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f808200d640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f807c06b180 con 0x7f807c065f60 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f808200d640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c06b440 con 0x7f807c06f400 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f807bfff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c06f400 0x7f807c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f807affd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f807c06c030 0x7f807c06ecf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34022/0 (socket says 172.21.15.133:34022) 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f807affd640 1 -- 172.21.15.133:0/4202427939 learned_addr learned my addr 172.21.15.133:0/4202427939 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f807bfff640 1 -- 172.21.15.133:0/4202427939 >> v1:172.21.15.191:6789/0 conn(0x7f807c065f60 legacy=0x7f807c06b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f807a7fc640 1 -- 172.21.15.133:0/4202427939 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343310770 0 0) 0x7f807c066340 con 0x7f807c06c030 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f807bfff640 1 -- 172.21.15.133:0/4202427939 >> v1:172.21.15.133:6789/0 conn(0x7f807c06c030 legacy=0x7f807c06ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.286+0000 7f807bfff640 1 -- 172.21.15.133:0/4202427939 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0b64d0 con 0x7f807c06f400 2024-09-16T15:07:30.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f807bfff640 1 --2- 172.21.15.133:0/4202427939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c06f400 0x7f807c0a3ab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8074009dc0 tx=0x7f807401c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d0fd77be1f1ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f807a7fc640 1 -- 172.21.15.133:0/4202427939 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740a5760 con 0x7f807c06f400 2024-09-16T15:07:30.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f807a7fc640 1 -- 172.21.15.133:0/4202427939 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80740a58c0 con 0x7f807c06f400 2024-09-16T15:07:30.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f807a7fc640 1 -- 172.21.15.133:0/4202427939 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740a5c50 con 0x7f807c06f400 2024-09-16T15:07:30.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 -- 172.21.15.133:0/4202427939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c06f400 msgr2=0x7f807c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 --2- 172.21.15.133:0/4202427939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c06f400 0x7f807c0a3ab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8074009dc0 tx=0x7f807401c8a0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 -- 172.21.15.133:0/4202427939 shutdown_connections 2024-09-16T15:07:30.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 --2- 172.21.15.133:0/4202427939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c06f400 0x7f807c0a3ab0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 -- 172.21.15.133:0/4202427939 >> 172.21.15.133:0/4202427939 conn(0x7f807c06a030 msgr2=0x7f807c06a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:30.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 -- 172.21.15.133:0/4202427939 shutdown_connections 2024-09-16T15:07:30.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 -- 172.21.15.133:0/4202427939 wait complete. 2024-09-16T15:07:30.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 Processor -- start 2024-09-16T15:07:30.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 -- start start 2024-09-16T15:07:30.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c065f60 0x7f807c1452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f807c06f400 0x7f807c14b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f807c14ba20 0x7f807c146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c148b80 con 0x7f807c06f400 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0a1bf0 con 0x7f807c14ba20 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f808200d640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c14a880 con 0x7f807c065f60 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f807affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f807c06f400 0x7f807c14b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c065f60 0x7f807c1452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f807affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f807c06f400 0x7f807c14b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57478/0 (socket says 172.21.15.133:57478) 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.290+0000 7f807affd640 1 -- 172.21.15.133:0/1679848849 learned_addr learned my addr 172.21.15.133:0/1679848849 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f807bfff640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f807c14ba20 0x7f807c146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f807affd640 1 -- 172.21.15.133:0/1679848849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c065f60 msgr2=0x7f807c1452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f807affd640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c065f60 0x7f807c1452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f807affd640 1 -- 172.21.15.133:0/1679848849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f807c14ba20 msgr2=0x7f807c146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f807affd640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f807c14ba20 0x7f807c146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f807affd640 1 -- 172.21.15.133:0/1679848849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8074009a70 con 0x7f807c06f400 2024-09-16T15:07:30.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f807affd640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f807c06f400 0x7f807c14b4e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f806400c4b0 tx=0x7f806400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f8073fff640 1 -- 172.21.15.133:0/1679848849 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80640074e0 con 0x7f807c06f400 2024-09-16T15:07:30.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f8073fff640 1 -- 172.21.15.133:0/1679848849 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8064007660 con 0x7f807c06f400 2024-09-16T15:07:30.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f808200d640 1 -- 172.21.15.133:0/1679848849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c148db0 con 0x7f807c06f400 2024-09-16T15:07:30.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f8073fff640 1 -- 172.21.15.133:0/1679848849 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8064004a30 con 0x7f807c06f400 2024-09-16T15:07:30.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f808200d640 1 -- 172.21.15.133:0/1679848849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c14ad00 con 0x7f807c06f400 2024-09-16T15:07:30.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f808200d640 1 -- 172.21.15.133:0/1679848849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c0672a0 con 0x7f807c06f400 2024-09-16T15:07:30.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f8073fff640 1 -- 172.21.15.133:0/1679848849 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f806401c3b0 con 0x7f807c06f400 2024-09-16T15:07:30.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f8073fff640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:30.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f8073fff640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:30.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f8073fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:30.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f8073fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:30.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f8073fff640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f805c097870 0x7f805c099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f8073fff640 1 -- 172.21.15.133:0/1679848849 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f80640bf500 con 0x7f807c06f400 2024-09-16T15:07:30.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.294+0000 7f807b7fe640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f805c097870 0x7f805c099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.298+0000 7f807b7fe640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f805c097870 0x7f805c099d30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f806c007fd0 tx=0x7f806c007e40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.298+0000 7f8073fff640 1 -- 172.21.15.133:0/1679848849 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f806408cc10 con 0x7f807c06f400 2024-09-16T15:07:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.446+0000 7f808200d640 1 -- 172.21.15.133:0/1679848849 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f807c0a3ab0 con 0x7f807c06f400 2024-09-16T15:07:30.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.446+0000 7f8073fff640 1 -- 172.21.15.133:0/1679848849 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+13441 (secure 0 0 0) 0x7f8064018020 con 0x7f807c06f400 2024-09-16T15:07:30.453 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:30.453 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":22,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:27.225430+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:07:20.523939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-16T15:07:21.043356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:30.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 10 mgrc shutdown 2024-09-16T15:07:30.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 -- 172.21.15.133:0/1679848849 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f805c097870 msgr2=0x7f805c099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f805c097870 0x7f805c099d30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f806c007fd0 tx=0x7f806c007e40 comp rx=0 tx=0).stop 2024-09-16T15:07:30.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 -- 172.21.15.133:0/1679848849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f807c06f400 msgr2=0x7f807c14b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f807c06f400 0x7f807c14b4e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f806400c4b0 tx=0x7f806400c970 comp rx=0 tx=0).stop 2024-09-16T15:07:30.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 -- 172.21.15.133:0/1679848849 shutdown_connections 2024-09-16T15:07:30.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f807c14ba20 0x7f807c146bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f805c097870 0x7f805c099d30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f807c06f400 0x7f807c14b4e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 --2- 172.21.15.133:0/1679848849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f807c065f60 0x7f807c1452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 -- 172.21.15.133:0/1679848849 >> 172.21.15.133:0/1679848849 conn(0x7f807c06a030 msgr2=0x7f807c0b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:30.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 -- 172.21.15.133:0/1679848849 shutdown_connections 2024-09-16T15:07:30.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.450+0000 7f808200d640 1 -- 172.21.15.133:0/1679848849 wait complete. 2024-09-16T15:07:30.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:07:08.758832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T15:07:14.143631+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T15:07:20.523939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T15:07:21.043356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:07:30.471 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:07:30.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.582+0000 7f4533424640 1 Processor -- start 2024-09-16T15:07:30.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.582+0000 7f4533424640 1 -- start start 2024-09-16T15:07:30.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.582+0000 7f4533424640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f452c0a9ae0 0x7f452c0a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4533424640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c139b60 con 0x7f452c1365b0 2024-09-16T15:07:30.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4533424640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c0a5ca0 con 0x7f452c0ab810 2024-09-16T15:07:30.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4533424640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c136990 con 0x7f452c0a9ae0 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45311c0640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f452c1365b0 0x7f452c0aa960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34024/0 (socket says 172.21.15.133:34024) 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45311c0640 1 -- 172.21.15.133:0/1200107019 learned_addr learned my addr 172.21.15.133:0/1200107019 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45309bf640 1 --2- 172.21.15.133:0/1200107019 >> v2:172.21.15.191:3300/0 conn(0x7f452c0a9ae0 0x7f452c0a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628749748 0 0) 0x7f452c139b60 con 0x7f452c1365b0 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4518003530 con 0x7f452c1365b0 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2712963923 0 0) 0x7f4518003530 con 0x7f452c1365b0 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 >> v2:172.21.15.191:3300/0 conn(0x7f452c0a9ae0 msgr2=0x7f452c0a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 --2- 172.21.15.133:0/1200107019 >> v2:172.21.15.191:3300/0 conn(0x7f452c0a9ae0 0x7f452c0a80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 >> v1:172.21.15.191:6789/0 conn(0x7f452c0ab810 legacy=0x7f452c14dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c0ab070 con 0x7f452c1365b0 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f451c001680 con 0x7f452c1365b0 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 >> v1:172.21.15.133:6789/0 conn(0x7f452c1365b0 legacy=0x7f452c0aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f4518003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45180047d0 0x7f4518004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45180052b0 0x7f451800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c136990 con 0x7f4518003980 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451800c180 con 0x7f45180047d0 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451800c340 con 0x7f45180052b0 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45311c0640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f4518003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45319c1640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45180052b0 0x7f451800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45309bf640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45180047d0 0x7f4518004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45311c0640 1 -- 172.21.15.133:0/1200107019 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45180052b0 msgr2=0x7f451800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45311c0640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45180052b0 0x7f451800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45311c0640 1 -- 172.21.15.133:0/1200107019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45180047d0 msgr2=0x7f4518004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45311c0640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45180047d0 0x7f4518004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45311c0640 1 -- 172.21.15.133:0/1200107019 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4518003530 con 0x7f4518003980 2024-09-16T15:07:30.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45309bf640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45180047d0 0x7f4518004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:07:30.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.586+0000 7f45319c1640 1 -- 172.21.15.133:0/1200107019 reap_dead start 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45311c0640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f4518003da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f451c00c210 tx=0x7f451c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ded2d83a4f7a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c005ae0 con 0x7f4518003980 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c005c40 con 0x7f4518003980 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4523fff640 1 -- 172.21.15.133:0/1200107019 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0a59f0 con 0x7f4518003980 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- 172.21.15.133:0/1200107019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 msgr2=0x7f4518003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f4518003da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f451c00c210 tx=0x7f451c01cbc0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- 172.21.15.133:0/1200107019 shutdown_connections 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 --2- 172.21.15.133:0/1200107019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f4518003da0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- 172.21.15.133:0/1200107019 >> 172.21.15.133:0/1200107019 conn(0x7f452c06a030 msgr2=0x7f452c06bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- 172.21.15.133:0/1200107019 shutdown_connections 2024-09-16T15:07:30.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- 172.21.15.133:0/1200107019 wait complete. 2024-09-16T15:07:30.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 Processor -- start 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- start start 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f452c131b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f452c1320d0 0x7f452c132f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c1334d0 0x7f452c133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0a5950 con 0x7f4518003980 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0a5120 con 0x7f452c1334d0 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c134c00 con 0x7f452c1320d0 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45311c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f452c131b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45309bf640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f452c1320d0 0x7f452c132f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45311c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f452c131b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57500/0 (socket says 172.21.15.133:57500) 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45311c0640 1 -- 172.21.15.133:0/389857098 learned_addr learned my addr 172.21.15.133:0/389857098 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45319c1640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c1334d0 0x7f452c133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45311c0640 1 -- 172.21.15.133:0/389857098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f452c1320d0 msgr2=0x7f452c132f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45311c0640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f452c1320d0 0x7f452c132f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45311c0640 1 -- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c1334d0 msgr2=0x7f452c133f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45311c0640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c1334d0 0x7f452c133f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45311c0640 1 -- 172.21.15.133:0/389857098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c134ec0 con 0x7f4518003980 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45319c1640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c1334d0 0x7f452c133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f45311c0640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f452c131b90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f451c01c9f0 tx=0x7f4518005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4521ffb640 1 -- 172.21.15.133:0/389857098 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0b4070 con 0x7f4518003980 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4521ffb640 1 -- 172.21.15.133:0/389857098 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c0a59c0 con 0x7f4518003980 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- 172.21.15.133:0/389857098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c0b8270 con 0x7f4518003980 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4521ffb640 1 -- 172.21.15.133:0/389857098 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0a5d70 con 0x7f4518003980 2024-09-16T15:07:30.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.590+0000 7f4533424640 1 -- 172.21.15.133:0/389857098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c135660 con 0x7f4518003980 2024-09-16T15:07:30.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.594+0000 7f4533424640 1 -- 172.21.15.133:0/389857098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c0672a0 con 0x7f4518003980 2024-09-16T15:07:30.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.594+0000 7f4521ffb640 1 -- 172.21.15.133:0/389857098 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f451c007070 con 0x7f4518003980 2024-09-16T15:07:30.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.594+0000 7f4521ffb640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:30.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.594+0000 7f4521ffb640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:30.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.594+0000 7f4521ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:30.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.594+0000 7f4521ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:30.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.594+0000 7f4521ffb640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4510097840 0x7f4510099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.594+0000 7f4521ffb640 1 -- 172.21.15.133:0/389857098 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f451c159c80 con 0x7f4518003980 2024-09-16T15:07:30.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.594+0000 7f45309bf640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4510097840 0x7f4510099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.598+0000 7f45309bf640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4510097840 0x7f4510099d00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f452c0a8070 tx=0x7f4528008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.598+0000 7f4521ffb640 1 -- 172.21.15.133:0/389857098 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f451c127390 con 0x7f4518003980 2024-09-16T15:07:30.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 -- 172.21.15.133:0/389857098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f452c132c90 con 0x7f4518003980 2024-09-16T15:07:30.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4521ffb640 1 -- 172.21.15.133:0/389857098 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f451c1292c0 con 0x7f4518003980 2024-09-16T15:07:30.758 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 10 mgrc shutdown 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 -- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4510097840 msgr2=0x7f4510099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4510097840 0x7f4510099d00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f452c0a8070 tx=0x7f4528008040 comp rx=0 tx=0).stop 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 -- 172.21.15.133:0/389857098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 msgr2=0x7f452c131b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f452c131b90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f451c01c9f0 tx=0x7f4518005800 comp rx=0 tx=0).stop 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 -- 172.21.15.133:0/389857098 shutdown_connections 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f452c1334d0 0x7f452c133f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f452c1320d0 0x7f452c132f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4510097840 0x7f4510099d00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 --2- 172.21.15.133:0/389857098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4518003980 0x7f452c131b90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 -- 172.21.15.133:0/389857098 >> 172.21.15.133:0/389857098 conn(0x7f452c06a030 msgr2=0x7f452c138530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:30.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 -- 172.21.15.133:0/389857098 shutdown_connections 2024-09-16T15:07:30.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.754+0000 7f4533424640 1 -- 172.21.15.133:0/389857098 wait complete. 2024-09-16T15:07:30.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-16T15:07:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20aa413640 1 Processor -- start 2024-09-16T15:07:30.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20aa413640 1 -- start start 2024-09-16T15:07:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20aa413640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20a41412e0 0x7f20a409edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20aa413640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f20a40a2fe0 0x7f20a414aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20aa413640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a409f310 con 0x7f20a40a6210 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20aa413640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a40a3590 con 0x7f20a41412e0 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20aa413640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a40a3850 con 0x7f20a40a2fe0 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20a41412e0 0x7f20a409edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a37fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f20a41412e0 0x7f20a409edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57512/0 (socket says 172.21.15.133:57512) 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a37fe640 1 -- 172.21.15.133:0/151347467 learned_addr learned my addr 172.21.15.133:0/151347467 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a89b0640 1 --2- 172.21.15.133:0/151347467 >> v2:172.21.15.191:3300/0 conn(0x7f20a40a2fe0 0x7f20a414aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a37fe640 1 -- 172.21.15.133:0/151347467 >> v2:172.21.15.191:3300/0 conn(0x7f20a40a2fe0 msgr2=0x7f20a414aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a37fe640 1 --2- 172.21.15.133:0/151347467 >> v2:172.21.15.191:3300/0 conn(0x7f20a40a2fe0 0x7f20a414aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a37fe640 1 -- 172.21.15.133:0/151347467 >> v1:172.21.15.133:6789/0 conn(0x7f20a40a6210 legacy=0x7f20a4140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a2ffd640 1 -- 172.21.15.133:0/151347467 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76421579 0 0) 0x7f2094001120 con 0x7f20a40a6210 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a37fe640 1 -- 172.21.15.133:0/151347467 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a40a65f0 con 0x7f20a41412e0 2024-09-16T15:07:30.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a89b0640 1 --2- 172.21.15.133:0/151347467 >> v2:172.21.15.191:3300/0 conn(0x7f20a40a2fe0 0x7f20a414aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a37fe640 1 --2- 172.21.15.133:0/151347467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a41412e0 0x7f20a409edd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f20a40a5960 tx=0x7f209801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f884f863618e8851 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a2ffd640 1 -- 172.21.15.133:0/151347467 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980ab600 con 0x7f20a41412e0 2024-09-16T15:07:30.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a2ffd640 1 -- 172.21.15.133:0/151347467 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20980a7050 con 0x7f20a41412e0 2024-09-16T15:07:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.886+0000 7f20a2ffd640 1 -- 172.21.15.133:0/151347467 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980ab980 con 0x7f20a41412e0 2024-09-16T15:07:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- 172.21.15.133:0/151347467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a41412e0 msgr2=0x7f20a409edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 --2- 172.21.15.133:0/151347467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a41412e0 0x7f20a409edd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f20a40a5960 tx=0x7f209801c3e0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- 172.21.15.133:0/151347467 shutdown_connections 2024-09-16T15:07:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 --2- 172.21.15.133:0/151347467 >> v2:172.21.15.191:3300/0 conn(0x7f20a40a2fe0 0x7f20a414aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 --2- 172.21.15.133:0/151347467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a41412e0 0x7f20a409edd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- 172.21.15.133:0/151347467 >> 172.21.15.133:0/151347467 conn(0x7f20a40a3fd0 msgr2=0x7f20a40a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:30.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- 172.21.15.133:0/151347467 shutdown_connections 2024-09-16T15:07:30.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- 172.21.15.133:0/151347467 wait complete. 2024-09-16T15:07:30.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 Processor -- start 2024-09-16T15:07:30.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- start start 2024-09-16T15:07:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a40a2fe0 0x7f20a40b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20a40a6210 0x7f20a413e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a41412e0 0x7f20a413ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a40b2c80 con 0x7f20a40a2fe0 2024-09-16T15:07:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a4139270 con 0x7f20a40a6210 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a40b5b20 con 0x7f20a41412e0 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a40a2fe0 0x7f20a40b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a89b0640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a41412e0 0x7f20a413ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a89b0640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a41412e0 0x7f20a413ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53124/0 (socket says 172.21.15.133:53124) 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a40a2fe0 0x7f20a40b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57526/0 (socket says 172.21.15.133:57526) 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a89b0640 1 -- 172.21.15.133:0/3791359328 learned_addr learned my addr 172.21.15.133:0/3791359328 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a37fe640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20a40a6210 0x7f20a413e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a3fff640 1 -- 172.21.15.133:0/3791359328 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a41412e0 msgr2=0x7f20a413ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a3fff640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a41412e0 0x7f20a413ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a3fff640 1 -- 172.21.15.133:0/3791359328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20a40a6210 msgr2=0x7f20a413e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a3fff640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20a40a6210 0x7f20a413e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a3fff640 1 -- 172.21.15.133:0/3791359328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20980094d0 con 0x7f20a40a2fe0 2024-09-16T15:07:30.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a89b0640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a41412e0 0x7f20a413ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a3fff640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a40a2fe0 0x7f20a40b55e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f209400c780 tx=0x7f209400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a0ff9640 1 -- 172.21.15.133:0/3791359328 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094005070 con 0x7f20a40a2fe0 2024-09-16T15:07:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a0ff9640 1 -- 172.21.15.133:0/3791359328 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20940051d0 con 0x7f20a40a2fe0 2024-09-16T15:07:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- 172.21.15.133:0/3791359328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a40b2eb0 con 0x7f20a40a2fe0 2024-09-16T15:07:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20a0ff9640 1 -- 172.21.15.133:0/3791359328 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209400a640 con 0x7f20a40a2fe0 2024-09-16T15:07:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- 172.21.15.133:0/3791359328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a4138370 con 0x7f20a40a2fe0 2024-09-16T15:07:30.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.890+0000 7f20aa413640 1 -- 172.21.15.133:0/3791359328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a40a2c50 con 0x7f20a40a2fe0 2024-09-16T15:07:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.894+0000 7f20a0ff9640 1 -- 172.21.15.133:0/3791359328 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f209400a7a0 con 0x7f20a40a2fe0 2024-09-16T15:07:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.894+0000 7f20a0ff9640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.894+0000 7f20a0ff9640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.894+0000 7f20a0ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.894+0000 7f20a0ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.894+0000 7f20a0ff9640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2090097840 0x7f2090099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.894+0000 7f20a0ff9640 1 -- 172.21.15.133:0/3791359328 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f20940bf3d0 con 0x7f20a40a2fe0 2024-09-16T15:07:30.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.894+0000 7f20a37fe640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2090097840 0x7f2090099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:30.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.894+0000 7f20a37fe640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2090097840 0x7f2090099d00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f20980ad000 tx=0x7f20980a6000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:30.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:30.898+0000 7f20a0ff9640 1 -- 172.21.15.133:0/3791359328 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f209408cb70 con 0x7f20a40a2fe0 2024-09-16T15:07:31.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.038+0000 7f20aa413640 1 -- 172.21.15.133:0/3791359328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f20a409f310 con 0x7f20a40a2fe0 2024-09-16T15:07:31.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20a0ff9640 1 -- 172.21.15.133:0/3791359328 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f20a409f310 con 0x7f20a40a2fe0 2024-09-16T15:07:31.045 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 8 2024-09-16T15:07:31.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 10 mgrc shutdown 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 -- 172.21.15.133:0/3791359328 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2090097840 msgr2=0x7f2090099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2090097840 0x7f2090099d00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f20980ad000 tx=0x7f20980a6000 comp rx=0 tx=0).stop 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 -- 172.21.15.133:0/3791359328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a40a2fe0 msgr2=0x7f20a40b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a40a2fe0 0x7f20a40b55e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f209400c780 tx=0x7f209400cc40 comp rx=0 tx=0).stop 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 -- 172.21.15.133:0/3791359328 shutdown_connections 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20a41412e0 0x7f20a413ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20a40a6210 0x7f20a413e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2090097840 0x7f2090099d00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 --2- 172.21.15.133:0/3791359328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20a40a2fe0 0x7f20a40b55e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 -- 172.21.15.133:0/3791359328 >> 172.21.15.133:0/3791359328 conn(0x7f20a40a3fd0 msgr2=0x7f20a413c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 -- 172.21.15.133:0/3791359328 shutdown_connections 2024-09-16T15:07:31.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.042+0000 7f20aa413640 1 -- 172.21.15.133:0/3791359328 wait complete. 2024-09-16T15:07:31.062 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T15:07:31.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f928a081640 1 Processor -- start 2024-09-16T15:07:31.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f928a081640 1 -- start start 2024-09-16T15:07:31.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f928a081640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f92841412e0 0x7f928409edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f928a081640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f928414aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f928a081640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928409f310 con 0x7f92840a6210 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f928a081640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92840a3590 con 0x7f92841412e0 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f928a081640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92840a3850 con 0x7f92840a2fe0 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f9283fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f928414aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f9283fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f928414aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53132/0 (socket says 172.21.15.133:53132) 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f9283fff640 1 -- 172.21.15.133:0/1244902139 learned_addr learned my addr 172.21.15.133:0/1244902139 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f9282ffd640 1 --2- 172.21.15.133:0/1244902139 >> v2:172.21.15.191:3300/0 conn(0x7f92841412e0 0x7f928409edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f9283fff640 1 -- 172.21.15.133:0/1244902139 >> v2:172.21.15.191:3300/0 conn(0x7f92841412e0 msgr2=0x7f928409edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f9283fff640 1 --2- 172.21.15.133:0/1244902139 >> v2:172.21.15.191:3300/0 conn(0x7f92841412e0 0x7f928409edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f9283fff640 1 -- 172.21.15.133:0/1244902139 >> v1:172.21.15.191:6789/0 conn(0x7f92840a6210 legacy=0x7f9284140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f9283fff640 1 -- 172.21.15.133:0/1244902139 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840a65f0 con 0x7f92840a2fe0 2024-09-16T15:07:31.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.166+0000 7f9282ffd640 1 --2- 172.21.15.133:0/1244902139 >> v2:172.21.15.191:3300/0 conn(0x7f92841412e0 0x7f928409edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:31.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f9283fff640 1 --2- 172.21.15.133:0/1244902139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f928414aca0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9274009870 tx=0x7f927401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d62a49540bc29a73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:31.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f92827fc640 1 -- 172.21.15.133:0/1244902139 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92740ab600 con 0x7f92840a2fe0 2024-09-16T15:07:31.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f92827fc640 1 -- 172.21.15.133:0/1244902139 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92740a7050 con 0x7f92840a2fe0 2024-09-16T15:07:31.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f92827fc640 1 -- 172.21.15.133:0/1244902139 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92740ab980 con 0x7f92840a2fe0 2024-09-16T15:07:31.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f928a081640 1 -- 172.21.15.133:0/1244902139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 msgr2=0x7f928414aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f928a081640 1 --2- 172.21.15.133:0/1244902139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f928414aca0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9274009870 tx=0x7f927401c3e0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f928a081640 1 -- 172.21.15.133:0/1244902139 shutdown_connections 2024-09-16T15:07:31.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f928a081640 1 --2- 172.21.15.133:0/1244902139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f928414aca0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f928a081640 1 --2- 172.21.15.133:0/1244902139 >> v2:172.21.15.191:3300/0 conn(0x7f92841412e0 0x7f928409edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f928a081640 1 -- 172.21.15.133:0/1244902139 >> 172.21.15.133:0/1244902139 conn(0x7f92840a3fd0 msgr2=0x7f92840a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:31.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f928a081640 1 -- 172.21.15.133:0/1244902139 shutdown_connections 2024-09-16T15:07:31.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f928a081640 1 -- 172.21.15.133:0/1244902139 wait complete. 2024-09-16T15:07:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f928a081640 1 Processor -- start 2024-09-16T15:07:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.170+0000 7f928a081640 1 -- start start 2024-09-16T15:07:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f928a081640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f92841392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f928a081640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92840a6210 0x7f92841deb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f928a081640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92841412e0 0x7f92841df080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f928a081640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92841df730 con 0x7f92840a6210 2024-09-16T15:07:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f928a081640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92841df9f0 con 0x7f92841412e0 2024-09-16T15:07:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f928a081640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92840b33d0 con 0x7f92840a2fe0 2024-09-16T15:07:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f92837fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f92841392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f9282ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92840a6210 0x7f92841deb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f92837fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f92841392b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53140/0 (socket says 172.21.15.133:53140) 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f92837fe640 1 -- 172.21.15.133:0/3282906575 learned_addr learned my addr 172.21.15.133:0/3282906575 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f92837fe640 1 -- 172.21.15.133:0/3282906575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92841412e0 msgr2=0x7f92841df080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f9283fff640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92841412e0 0x7f92841df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f92837fe640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92841412e0 0x7f92841df080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f92837fe640 1 -- 172.21.15.133:0/3282906575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92840a6210 msgr2=0x7f92841deb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f92837fe640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92840a6210 0x7f92841deb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f92837fe640 1 -- 172.21.15.133:0/3282906575 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92740094d0 con 0x7f92840a2fe0 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f92837fe640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f92841392b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f927c00c7c0 tx=0x7f927c00cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f927bfff640 1 -- 172.21.15.133:0/3282906575 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00aa50 con 0x7f92840a2fe0 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f927bfff640 1 -- 172.21.15.133:0/3282906575 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c00abb0 con 0x7f92840a2fe0 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f928a081640 1 -- 172.21.15.133:0/3282906575 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928409d8d0 con 0x7f92840a2fe0 2024-09-16T15:07:31.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f927bfff640 1 -- 172.21.15.133:0/3282906575 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c012420 con 0x7f92840a2fe0 2024-09-16T15:07:31.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f928a081640 1 -- 172.21.15.133:0/3282906575 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92840b5b50 con 0x7f92840a2fe0 2024-09-16T15:07:31.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f928a081640 1 -- 172.21.15.133:0/3282906575 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928409edd0 con 0x7f92840a2fe0 2024-09-16T15:07:31.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f927bfff640 1 -- 172.21.15.133:0/3282906575 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f927c006b60 con 0x7f92840a2fe0 2024-09-16T15:07:31.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f927bfff640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:31.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f927bfff640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:31.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f927bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:31.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f927bfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:31.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f927bfff640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9264097870 0x7f9264099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f927bfff640 1 -- 172.21.15.133:0/3282906575 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f927c0c0780 con 0x7f92840a2fe0 2024-09-16T15:07:31.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.174+0000 7f9282ffd640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9264097870 0x7f9264099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.178+0000 7f9282ffd640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9264097870 0x7f9264099d30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f92840a6d30 tx=0x7f926c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:31.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.178+0000 7f927bfff640 1 -- 172.21.15.133:0/3282906575 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f927c08de90 con 0x7f92840a2fe0 2024-09-16T15:07:31.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 -- 172.21.15.133:0/3282906575 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f928406ba00 con 0x7f92840a2fe0 2024-09-16T15:07:31.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f927bfff640 1 -- 172.21.15.133:0/3282906575 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f927c08fdc0 con 0x7f92840a2fe0 2024-09-16T15:07:31.330 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 10 mgrc shutdown 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 -- 172.21.15.133:0/3282906575 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9264097870 msgr2=0x7f9264099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9264097870 0x7f9264099d30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f92840a6d30 tx=0x7f926c008040 comp rx=0 tx=0).stop 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 -- 172.21.15.133:0/3282906575 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 msgr2=0x7f92841392b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f92841392b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f927c00c7c0 tx=0x7f927c00cc80 comp rx=0 tx=0).stop 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 -- 172.21.15.133:0/3282906575 shutdown_connections 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92841412e0 0x7f92841df080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9264097870 0x7f9264099d30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92840a6210 0x7f92841deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 --2- 172.21.15.133:0/3282906575 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92840a2fe0 0x7f92841392b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 -- 172.21.15.133:0/3282906575 >> 172.21.15.133:0/3282906575 conn(0x7f92840a3fd0 msgr2=0x7f92841429d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 -- 172.21.15.133:0/3282906575 shutdown_connections 2024-09-16T15:07:31.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.326+0000 7f928a081640 1 -- 172.21.15.133:0/3282906575 wait complete. 2024-09-16T15:07:31.348 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T15:07:31.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.458+0000 7f65d46f5640 1 Processor -- start 2024-09-16T15:07:31.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d46f5640 1 -- start start 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d46f5640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc0a2cf0 0x7f65cc0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d46f5640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f65cc0a1010 0x7f65cc14bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d46f5640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65cc06a8f0 con 0x7f65cc09e490 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d46f5640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65cc0a5bd0 con 0x7f65cc0a1010 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d46f5640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc0a5910 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d2491640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc0a2cf0 0x7f65cc0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d2491640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc0a2cf0 0x7f65cc0a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53144/0 (socket says 172.21.15.133:53144) 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d2491640 1 -- 172.21.15.133:0/3111651150 learned_addr learned my addr 172.21.15.133:0/3111651150 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d2491640 1 -- 172.21.15.133:0/3111651150 >> v2:172.21.15.191:3300/0 conn(0x7f65cc0a1010 msgr2=0x7f65cc14bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d2491640 1 --2- 172.21.15.133:0/3111651150 >> v2:172.21.15.191:3300/0 conn(0x7f65cc0a1010 0x7f65cc14bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d2491640 1 -- 172.21.15.133:0/3111651150 >> v1:172.21.15.191:6789/0 conn(0x7f65cc09e490 legacy=0x7f65cc06afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:31.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d2491640 1 -- 172.21.15.133:0/3111651150 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc0a4cf0 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d2491640 1 --2- 172.21.15.133:0/3111651150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc0a2cf0 0x7f65cc0a53d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009870 tx=0x7f65c801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a48e01b8bbf09ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:31.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d148f640 1 -- 172.21.15.133:0/3111651150 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80ac600 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d148f640 1 -- 172.21.15.133:0/3111651150 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80a8050 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d148f640 1 -- 172.21.15.133:0/3111651150 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80ac980 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d46f5640 1 -- 172.21.15.133:0/3111651150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc0a2cf0 msgr2=0x7f65cc0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d46f5640 1 --2- 172.21.15.133:0/3111651150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc0a2cf0 0x7f65cc0a53d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009870 tx=0x7f65c801c3e0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.462+0000 7f65d46f5640 1 -- 172.21.15.133:0/3111651150 shutdown_connections 2024-09-16T15:07:31.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 --2- 172.21.15.133:0/3111651150 >> v2:172.21.15.191:3300/0 conn(0x7f65cc0a1010 0x7f65cc14bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 --2- 172.21.15.133:0/3111651150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc0a2cf0 0x7f65cc0a53d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 -- 172.21.15.133:0/3111651150 >> 172.21.15.133:0/3111651150 conn(0x7f65cc0a3fd0 msgr2=0x7f65cc0a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:31.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 -- 172.21.15.133:0/3111651150 shutdown_connections 2024-09-16T15:07:31.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 -- 172.21.15.133:0/3111651150 wait complete. 2024-09-16T15:07:31.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 Processor -- start 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 -- start start 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc09e490 0x7f65cc1411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0a1010 0x7f65cc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65cc0a2cf0 0x7f65cc1dd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc141c00 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1dd6e0 con 0x7f65cc0a1010 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1dd9a0 con 0x7f65cc09e490 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2491640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc09e490 0x7f65cc1411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2c92640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65cc0a2cf0 0x7f65cc1dd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2491640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc09e490 0x7f65cc1411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53158/0 (socket says 172.21.15.133:53158) 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2491640 1 -- 172.21.15.133:0/377573065 learned_addr learned my addr 172.21.15.133:0/377573065 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d1c90640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0a1010 0x7f65cc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2c92640 1 -- 172.21.15.133:0/377573065 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc09e490 msgr2=0x7f65cc1411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.472 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2c92640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc09e490 0x7f65cc1411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2c92640 1 -- 172.21.15.133:0/377573065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0a1010 msgr2=0x7f65cc06ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2c92640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0a1010 0x7f65cc06ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2c92640 1 -- 172.21.15.133:0/377573065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80094d0 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2491640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc09e490 0x7f65cc1411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:31.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d2c92640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65cc0a2cf0 0x7f65cc1dd000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65c400be20 tx=0x7f65c400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:31.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65c2ffd640 1 -- 172.21.15.133:0/377573065 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c400ce70 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65c2ffd640 1 -- 172.21.15.133:0/377573065 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c4014440 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65c2ffd640 1 -- 172.21.15.133:0/377573065 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4014810 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 -- 172.21.15.133:0/377573065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc06b870 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 -- 172.21.15.133:0/377573065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc071e40 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.466+0000 7f65d46f5640 1 -- 172.21.15.133:0/377573065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65cc06e580 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.470+0000 7f65c2ffd640 1 -- 172.21.15.133:0/377573065 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f65c401d3b0 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.470+0000 7f65c2ffd640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:31.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.470+0000 7f65c2ffd640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:31.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.470+0000 7f65c2ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:31.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.470+0000 7f65c2ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:31.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.470+0000 7f65c2ffd640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f65b0097840 0x7f65b0099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.470+0000 7f65c2ffd640 1 -- 172.21.15.133:0/377573065 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f65c40c00a0 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.470+0000 7f65d2491640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f65b0097840 0x7f65b0099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.470+0000 7f65d2491640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f65b0097840 0x7f65b0099d00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65c80af040 tx=0x7f65c80a7000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:31.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.474+0000 7f65c2ffd640 1 -- 172.21.15.133:0/377573065 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f65c408d860 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.618+0000 7f65d46f5640 1 -- 172.21.15.133:0/377573065 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65cc06e580 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.618+0000 7f65c2ffd640 1 -- 172.21.15.133:0/377573065 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65cc06e580 con 0x7f65cc0a2cf0 2024-09-16T15:07:31.623 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-09-16T15:07:31.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 10 mgrc shutdown 2024-09-16T15:07:31.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 -- 172.21.15.133:0/377573065 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f65b0097840 msgr2=0x7f65b0099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f65b0097840 0x7f65b0099d00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65c80af040 tx=0x7f65c80a7000 comp rx=0 tx=0).stop 2024-09-16T15:07:31.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 -- 172.21.15.133:0/377573065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65cc0a2cf0 msgr2=0x7f65cc1dd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65cc0a2cf0 0x7f65cc1dd000 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65c400be20 tx=0x7f65c400bf10 comp rx=0 tx=0).stop 2024-09-16T15:07:31.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 -- 172.21.15.133:0/377573065 shutdown_connections 2024-09-16T15:07:31.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f65b0097840 0x7f65b0099d00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65cc0a2cf0 0x7f65cc1dd000 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0a1010 0x7f65cc06ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 --2- 172.21.15.133:0/377573065 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65cc09e490 0x7f65cc1411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 -- 172.21.15.133:0/377573065 >> 172.21.15.133:0/377573065 conn(0x7f65cc0a3fd0 msgr2=0x7f65cc06adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:31.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 -- 172.21.15.133:0/377573065 shutdown_connections 2024-09-16T15:07:31.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.622+0000 7f65d46f5640 1 -- 172.21.15.133:0/377573065 wait complete. 2024-09-16T15:07:31.641 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:07:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf8476640 1 Processor -- start 2024-09-16T15:07:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf8476640 1 -- start start 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf8476640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1cf01365b0 0x7f1cf00aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf8476640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf00ab810 0x7f1cf014dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf8476640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf0139b60 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf8476640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf00a5ca0 con 0x7f1cf01365b0 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf8476640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf0136990 con 0x7f1cf00ab810 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf5a11640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1cf00a9ae0 0x7f1cf00a80d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34042/0 (socket says 172.21.15.133:34042) 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf5a11640 1 -- 172.21.15.133:0/2026852573 learned_addr learned my addr 172.21.15.133:0/2026852573 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf6a13640 1 --2- 172.21.15.133:0/2026852573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf00ab810 0x7f1cf014dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf6212640 1 --2- 172.21.15.133:0/2026852573 >> v2:172.21.15.191:3300/0 conn(0x7f1cf01365b0 0x7f1cf00aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf5210640 1 -- 172.21.15.133:0/2026852573 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147323742 0 0) 0x7f1cf0139b60 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf5210640 1 -- 172.21.15.133:0/2026852573 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cdc003530 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf6a13640 1 -- 172.21.15.133:0/2026852573 >> v2:172.21.15.191:3300/0 conn(0x7f1cf01365b0 msgr2=0x7f1cf00aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf6a13640 1 --2- 172.21.15.133:0/2026852573 >> v2:172.21.15.191:3300/0 conn(0x7f1cf01365b0 0x7f1cf00aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf6a13640 1 -- 172.21.15.133:0/2026852573 >> v1:172.21.15.133:6789/0 conn(0x7f1cf00a9ae0 legacy=0x7f1cf00a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf6a13640 1 -- 172.21.15.133:0/2026852573 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf00a6a60 con 0x7f1cf00ab810 2024-09-16T15:07:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf6212640 1 --2- 172.21.15.133:0/2026852573 >> v2:172.21.15.191:3300/0 conn(0x7f1cf01365b0 0x7f1cf00aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:07:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.758+0000 7f1cf6a13640 1 --2- 172.21.15.133:0/2026852573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf00ab810 0x7f1cf014dc40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009d90 tx=0x7f1ce801c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=608abbe26640a099 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf5210640 1 -- 172.21.15.133:0/2026852573 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80a5910 con 0x7f1cf00ab810 2024-09-16T15:07:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf5210640 1 -- 172.21.15.133:0/2026852573 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce80a5a70 con 0x7f1cf00ab810 2024-09-16T15:07:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf5210640 1 -- 172.21.15.133:0/2026852573 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80a5e00 con 0x7f1cf00ab810 2024-09-16T15:07:31.765 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- 172.21.15.133:0/2026852573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf00ab810 msgr2=0x7f1cf014dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 --2- 172.21.15.133:0/2026852573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf00ab810 0x7f1cf014dc40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009d90 tx=0x7f1ce801c830 comp rx=0 tx=0).stop 2024-09-16T15:07:31.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- 172.21.15.133:0/2026852573 shutdown_connections 2024-09-16T15:07:31.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 --2- 172.21.15.133:0/2026852573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf00ab810 0x7f1cf014dc40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 --2- 172.21.15.133:0/2026852573 >> v2:172.21.15.191:3300/0 conn(0x7f1cf01365b0 0x7f1cf00aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- 172.21.15.133:0/2026852573 >> 172.21.15.133:0/2026852573 conn(0x7f1cf006a030 msgr2=0x7f1cf006bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:31.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- 172.21.15.133:0/2026852573 shutdown_connections 2024-09-16T15:07:31.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- 172.21.15.133:0/2026852573 wait complete. 2024-09-16T15:07:31.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 Processor -- start 2024-09-16T15:07:31.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- start start 2024-09-16T15:07:31.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cf00a9ae0 0x7f1cf012fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cf00ab810 0x7f1cf009f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf01365b0 0x7f1cf01dbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf0130480 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf012ff80 con 0x7f1cf00ab810 2024-09-16T15:07:31.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf01301e0 con 0x7f1cf01365b0 2024-09-16T15:07:31.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf6212640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cf00a9ae0 0x7f1cf012fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf6a13640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf01365b0 0x7f1cf01dbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf6212640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cf00a9ae0 0x7f1cf012fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57542/0 (socket says 172.21.15.133:57542) 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf6212640 1 -- 172.21.15.133:0/1900571560 learned_addr learned my addr 172.21.15.133:0/1900571560 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf5a11640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cf00ab810 0x7f1cf009f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf6212640 1 -- 172.21.15.133:0/1900571560 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf01365b0 msgr2=0x7f1cf01dbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf6212640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf01365b0 0x7f1cf01dbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf6212640 1 -- 172.21.15.133:0/1900571560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cf00ab810 msgr2=0x7f1cf009f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf6212640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cf00ab810 0x7f1cf009f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf6212640 1 -- 172.21.15.133:0/1900571560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce8009a70 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf6212640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cf00a9ae0 0x7f1cf012fa40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00d800 tx=0x7f1cec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1ce6ffd640 1 -- 172.21.15.133:0/1900571560 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00b6c0 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1ce6ffd640 1 -- 172.21.15.133:0/1900571560 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec00b820 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- 172.21.15.133:0/1900571560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf01db6a0 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1ce6ffd640 1 -- 172.21.15.133:0/1900571560 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00bbd0 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.762+0000 7f1cf8476640 1 -- 172.21.15.133:0/1900571560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf0146c90 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.766+0000 7f1cf8476640 1 -- 172.21.15.133:0/1900571560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf00672a0 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.766+0000 7f1ce6ffd640 1 -- 172.21.15.133:0/1900571560 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f1cec00bd30 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.766+0000 7f1ce6ffd640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:31.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.766+0000 7f1ce6ffd640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:31.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.766+0000 7f1ce6ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:31.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.766+0000 7f1ce6ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:31.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.766+0000 7f1ce6ffd640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1cd4097840 0x7f1cd4099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:31.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.766+0000 7f1ce6ffd640 1 -- 172.21.15.133:0/1900571560 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f1cec0c0cf0 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.766+0000 7f1cf5a11640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1cd4097840 0x7f1cd4099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:31.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.766+0000 7f1cf5a11640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1cd4097840 0x7f1cd4099d00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1cf006c710 tx=0x7f1ce0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:31.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.770+0000 7f1ce6ffd640 1 -- 172.21.15.133:0/1900571560 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1cec08e360 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 -- 172.21.15.133:0/1900571560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cf00aa960 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1ce6ffd640 1 -- 172.21.15.133:0/1900571560 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3978 (secure 0 0 0) 0x7f1cec090290 con 0x7f1cf00a9ae0 2024-09-16T15:07:31.970 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:31.970 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6811/3104644688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3104644688},{"type":"v1","addr":"172.21.15.133:6811","nonce":3104644688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6809/147821810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":147821810},{"type":"v1","addr":"172.21.15.133:6809","nonce":147821810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:07:22.048013+0000","modified":"2024-09-16T15:07:24.720576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6827/2751563681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2751563681},{"type":"v1","addr":"172.21.15.191:6827","nonce":2751563681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-16T15:07:31.970 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-09-16T15:07:31.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 10 mgrc shutdown 2024-09-16T15:07:31.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 -- 172.21.15.133:0/1900571560 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1cd4097840 msgr2=0x7f1cd4099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1cd4097840 0x7f1cd4099d00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1cf006c710 tx=0x7f1ce0009040 comp rx=0 tx=0).stop 2024-09-16T15:07:31.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 -- 172.21.15.133:0/1900571560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cf00a9ae0 msgr2=0x7f1cf012fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:31.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cf00a9ae0 0x7f1cf012fa40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00d800 tx=0x7f1cec00dcc0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 -- 172.21.15.133:0/1900571560 shutdown_connections 2024-09-16T15:07:31.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cf01365b0 0x7f1cf01dbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1cf00ab810 0x7f1cf009f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f1cd4097840 0x7f1cd4099d00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 --2- 172.21.15.133:0/1900571560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cf00a9ae0 0x7f1cf012fa40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:31.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 -- 172.21.15.133:0/1900571560 >> 172.21.15.133:0/1900571560 conn(0x7f1cf006a030 msgr2=0x7f1cf00a6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:31.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 -- 172.21.15.133:0/1900571560 shutdown_connections 2024-09-16T15:07:31.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:31.966+0000 7f1cf8476640 1 -- 172.21.15.133:0/1900571560 wait complete. 2024-09-16T15:07:31.989 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:32.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.094+0000 7f3dc59bd640 1 Processor -- start 2024-09-16T15:07:32.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.094+0000 7f3dc59bd640 1 -- start start 2024-09-16T15:07:32.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3dc00a6210 0x7f3dc009edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a2fe0 0x7f3dc0140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc009f310 con 0x7f3dc0141110 2024-09-16T15:07:32.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc00a3590 con 0x7f3dc00a6210 2024-09-16T15:07:32.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00a3850 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbf7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3dc0141110 0x7f3dc0143970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34052/0 (socket says 172.21.15.133:34052) 2024-09-16T15:07:32.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbf7fe640 1 -- 172.21.15.133:0/1807619501 learned_addr learned my addr 172.21.15.133:0/1807619501 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:32.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbe7fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a2fe0 0x7f3dc0140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbeffd640 1 --2- 172.21.15.133:0/1807619501 >> v2:172.21.15.191:3300/0 conn(0x7f3dc00a6210 0x7f3dc009edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbdffb640 1 -- 172.21.15.133:0/1807619501 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236622242 0 0) 0x7f3db80014a0 con 0x7f3dc0141110 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbdffb640 1 -- 172.21.15.133:0/1807619501 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3da8003530 con 0x7f3dc0141110 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbe7fc640 1 -- 172.21.15.133:0/1807619501 >> v2:172.21.15.191:3300/0 conn(0x7f3dc00a6210 msgr2=0x7f3dc009edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbe7fc640 1 --2- 172.21.15.133:0/1807619501 >> v2:172.21.15.191:3300/0 conn(0x7f3dc00a6210 0x7f3dc009edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbe7fc640 1 -- 172.21.15.133:0/1807619501 >> v1:172.21.15.133:6789/0 conn(0x7f3dc0141110 legacy=0x7f3dc0143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbe7fc640 1 -- 172.21.15.133:0/1807619501 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc01415c0 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbeffd640 1 --2- 172.21.15.133:0/1807619501 >> v2:172.21.15.191:3300/0 conn(0x7f3dc00a6210 0x7f3dc009edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbe7fc640 1 --2- 172.21.15.133:0/1807619501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a2fe0 0x7f3dc0140bd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3db40097f0 tx=0x7f3db401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7df53986e705a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbdffb640 1 -- 172.21.15.133:0/1807619501 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db40ab600 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbdffb640 1 -- 172.21.15.133:0/1807619501 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db40a7050 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dbdffb640 1 -- 172.21.15.133:0/1807619501 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db40ab980 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 -- 172.21.15.133:0/1807619501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a2fe0 msgr2=0x7f3dc0140bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 --2- 172.21.15.133:0/1807619501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a2fe0 0x7f3dc0140bd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3db40097f0 tx=0x7f3db401c3f0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 -- 172.21.15.133:0/1807619501 shutdown_connections 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 --2- 172.21.15.133:0/1807619501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a2fe0 0x7f3dc0140bd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 --2- 172.21.15.133:0/1807619501 >> v2:172.21.15.191:3300/0 conn(0x7f3dc00a6210 0x7f3dc009edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 -- 172.21.15.133:0/1807619501 >> 172.21.15.133:0/1807619501 conn(0x7f3dc00a3fd0 msgr2=0x7f3dc00a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 -- 172.21.15.133:0/1807619501 shutdown_connections 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.098+0000 7f3dc59bd640 1 -- 172.21.15.133:0/1807619501 wait complete. 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 Processor -- start 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 -- start start 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dc00a2fe0 0x7f3dc009d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a6210 0x7f3dc0146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc0141110 0x7f3dc0145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0145ec0 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0147150 con 0x7f3dc0141110 2024-09-16T15:07:32.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0147410 con 0x7f3dc00a6210 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dbeffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dc00a2fe0 0x7f3dc009d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dbe7fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a6210 0x7f3dc0146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dbeffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dc00a2fe0 0x7f3dc009d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57552/0 (socket says 172.21.15.133:57552) 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dbeffd640 1 -- 172.21.15.133:0/364842301 learned_addr learned my addr 172.21.15.133:0/364842301 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dbeffd640 1 -- 172.21.15.133:0/364842301 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a6210 msgr2=0x7f3dc0146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dbeffd640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a6210 0x7f3dc0146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dbeffd640 1 -- 172.21.15.133:0/364842301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc0141110 msgr2=0x7f3dc0145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dbeffd640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc0141110 0x7f3dc0145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dbeffd640 1 -- 172.21.15.133:0/364842301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db4009420 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dbeffd640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dc00a2fe0 0x7f3dc009d0b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3db000e810 tx=0x7f3db000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3da7fff640 1 -- 172.21.15.133:0/364842301 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db000c8d0 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3da7fff640 1 -- 172.21.15.133:0/364842301 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db000ca30 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 -- 172.21.15.133:0/364842301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc0071f30 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3da7fff640 1 -- 172.21.15.133:0/364842301 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db000cde0 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 -- 172.21.15.133:0/364842301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc0071a70 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.102+0000 7f3dc59bd640 1 -- 172.21.15.133:0/364842301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc0140bd0 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.106+0000 7f3da7fff640 1 -- 172.21.15.133:0/364842301 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f3db0026430 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.106+0000 7f3da7fff640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.106+0000 7f3da7fff640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.106+0000 7f3da7fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.106+0000 7f3da7fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.106+0000 7f3da7fff640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f3dac097840 0x7f3dac099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.106+0000 7f3da7fff640 1 -- 172.21.15.133:0/364842301 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f3db00bfe90 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.106+0000 7f3dbe7fc640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f3dac097840 0x7f3dac099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.106+0000 7f3dbe7fc640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f3dac097840 0x7f3dac099d00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3db40ad000 tx=0x7f3db4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:32.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.106+0000 7f3da7fff640 1 -- 172.21.15.133:0/364842301 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3db008d630 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.262+0000 7f3dc59bd640 1 -- 172.21.15.133:0/364842301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dc00671f0 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.262+0000 7f3da7fff640 1 -- 172.21.15.133:0/364842301 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+13441 (secure 0 0 0) 0x7f3db0090c50 con 0x7f3dc00a2fe0 2024-09-16T15:07:32.267 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:32.267 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":22,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:27.225430+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:07:20.523939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-16T15:07:21.043356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:32.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 10 mgrc shutdown 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 -- 172.21.15.133:0/364842301 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f3dac097840 msgr2=0x7f3dac099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f3dac097840 0x7f3dac099d00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3db40ad000 tx=0x7f3db4008040 comp rx=0 tx=0).stop 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 -- 172.21.15.133:0/364842301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dc00a2fe0 msgr2=0x7f3dc009d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dc00a2fe0 0x7f3dc009d0b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3db000e810 tx=0x7f3db000ecd0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 -- 172.21.15.133:0/364842301 shutdown_connections 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc0141110 0x7f3dc0145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dc00a6210 0x7f3dc0146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f3dac097840 0x7f3dac099d00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 --2- 172.21.15.133:0/364842301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dc00a2fe0 0x7f3dc009d0b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 -- 172.21.15.133:0/364842301 >> 172.21.15.133:0/364842301 conn(0x7f3dc00a3fd0 msgr2=0x7f3dc00b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 -- 172.21.15.133:0/364842301 shutdown_connections 2024-09-16T15:07:32.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.266+0000 7f3dc59bd640 1 -- 172.21.15.133:0/364842301 wait complete. 2024-09-16T15:07:32.284 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:07:08.758832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '13', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-16T15:07:14.143631+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T15:07:20.523939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T15:07:21.043356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:07:32.285 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:07:32.329 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:07:32.331 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:07:32.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb13ed4640 1 Processor -- start 2024-09-16T15:07:32.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb13ed4640 1 -- start start 2024-09-16T15:07:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb13ed4640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c1365b0 0x7fdb0c0aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb13ed4640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdb0c0a9ae0 0x7fdb0c0a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb13ed4640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0c139b60 con 0x7fdb0c0ab810 2024-09-16T15:07:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb13ed4640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c0a5ca0 con 0x7fdb0c0a9ae0 2024-09-16T15:07:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb13ed4640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c136990 con 0x7fdb0c1365b0 2024-09-16T15:07:32.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb11c70640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c1365b0 0x7fdb0c0aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb11c70640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c1365b0 0x7fdb0c0aa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53212/0 (socket says 172.21.15.133:53212) 2024-09-16T15:07:32.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb11c70640 1 -- 172.21.15.133:0/2642542468 learned_addr learned my addr 172.21.15.133:0/2642542468 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:32.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb11c70640 1 -- 172.21.15.133:0/2642542468 >> v2:172.21.15.191:3300/0 conn(0x7fdb0c0a9ae0 msgr2=0x7fdb0c0a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:32.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb11c70640 1 --2- 172.21.15.133:0/2642542468 >> v2:172.21.15.191:3300/0 conn(0x7fdb0c0a9ae0 0x7fdb0c0a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb11c70640 1 -- 172.21.15.133:0/2642542468 >> v1:172.21.15.191:6789/0 conn(0x7fdb0c0ab810 legacy=0x7fdb0c0b9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:32.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.394+0000 7fdb11c70640 1 -- 172.21.15.133:0/2642542468 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0a6a60 con 0x7fdb0c1365b0 2024-09-16T15:07:32.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb11c70640 1 --2- 172.21.15.133:0/2642542468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c1365b0 0x7fdb0c0aa960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009870 tx=0x7fdb0801c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a2522a46f7783b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:32.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb10c6e640 1 -- 172.21.15.133:0/2642542468 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb080ab600 con 0x7fdb0c1365b0 2024-09-16T15:07:32.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb10c6e640 1 -- 172.21.15.133:0/2642542468 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb080a7050 con 0x7fdb0c1365b0 2024-09-16T15:07:32.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb10c6e640 1 -- 172.21.15.133:0/2642542468 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb080ab980 con 0x7fdb0c1365b0 2024-09-16T15:07:32.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 -- 172.21.15.133:0/2642542468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c1365b0 msgr2=0x7fdb0c0aa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 --2- 172.21.15.133:0/2642542468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c1365b0 0x7fdb0c0aa960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009870 tx=0x7fdb0801c430 comp rx=0 tx=0).stop 2024-09-16T15:07:32.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 -- 172.21.15.133:0/2642542468 shutdown_connections 2024-09-16T15:07:32.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 --2- 172.21.15.133:0/2642542468 >> v2:172.21.15.191:3300/0 conn(0x7fdb0c0a9ae0 0x7fdb0c0a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 --2- 172.21.15.133:0/2642542468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c1365b0 0x7fdb0c0aa960 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 -- 172.21.15.133:0/2642542468 >> 172.21.15.133:0/2642542468 conn(0x7fdb0c06a030 msgr2=0x7fdb0c06bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:32.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 -- 172.21.15.133:0/2642542468 shutdown_connections 2024-09-16T15:07:32.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 -- 172.21.15.133:0/2642542468 wait complete. 2024-09-16T15:07:32.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 Processor -- start 2024-09-16T15:07:32.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 -- start start 2024-09-16T15:07:32.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c0a9ae0 0x7fdb0c135160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0ab810 0x7fdb0c132580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb0c1365b0 0x7fdb0c0a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c1341b0 con 0x7fdb0c1365b0 2024-09-16T15:07:32.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c137d20 con 0x7fdb0c0ab810 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb13ed4640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c132fc0 con 0x7fdb0c0a9ae0 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb11c70640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c0a9ae0 0x7fdb0c135160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb12471640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb0c1365b0 0x7fdb0c0a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb11c70640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c0a9ae0 0x7fdb0c135160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53220/0 (socket says 172.21.15.133:53220) 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb11c70640 1 -- 172.21.15.133:0/1364811207 learned_addr learned my addr 172.21.15.133:0/1364811207 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.398+0000 7fdb1146f640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0ab810 0x7fdb0c132580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb12471640 1 -- 172.21.15.133:0/1364811207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c0a9ae0 msgr2=0x7fdb0c135160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb12471640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c0a9ae0 0x7fdb0c135160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb12471640 1 -- 172.21.15.133:0/1364811207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0ab810 msgr2=0x7fdb0c132580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb12471640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0ab810 0x7fdb0c132580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb12471640 1 -- 172.21.15.133:0/1364811207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb080094d0 con 0x7fdb0c1365b0 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb11c70640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c0a9ae0 0x7fdb0c135160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:32.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb12471640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb0c1365b0 0x7fdb0c0a2ae0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c780 tx=0x7fdb0400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:32.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb027fc640 1 -- 172.21.15.133:0/1364811207 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04003d90 con 0x7fdb0c1365b0 2024-09-16T15:07:32.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb027fc640 1 -- 172.21.15.133:0/1364811207 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0400ce30 con 0x7fdb0c1365b0 2024-09-16T15:07:32.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb13ed4640 1 -- 172.21.15.133:0/1364811207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c133280 con 0x7fdb0c1365b0 2024-09-16T15:07:32.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb027fc640 1 -- 172.21.15.133:0/1364811207 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04005290 con 0x7fdb0c1365b0 2024-09-16T15:07:32.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb13ed4640 1 -- 172.21.15.133:0/1364811207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c1349e0 con 0x7fdb0c1365b0 2024-09-16T15:07:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb13ed4640 1 -- 172.21.15.133:0/1364811207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c0672a0 con 0x7fdb0c1365b0 2024-09-16T15:07:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb027fc640 1 -- 172.21.15.133:0/1364811207 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7fdb04006b20 con 0x7fdb0c1365b0 2024-09-16T15:07:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb027fc640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb027fc640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb027fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb027fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb027fc640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdaf00977c0 0x7fdaf0099c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb027fc640 1 -- 172.21.15.133:0/1364811207 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7fdb040bf360 con 0x7fdb0c1365b0 2024-09-16T15:07:32.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.402+0000 7fdb11c70640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdaf00977c0 0x7fdaf0099c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.406+0000 7fdb11c70640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdaf00977c0 0x7fdaf0099c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb0801c470 tx=0x7fdb080a6000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:32.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.406+0000 7fdb027fc640 1 -- 172.21.15.133:0/1364811207 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdb0408ca70 con 0x7fdb0c1365b0 2024-09-16T15:07:32.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.562+0000 7fdb13ed4640 1 -- 172.21.15.133:0/1364811207 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdb0c0a80d0 con 0x7fdb0c1365b0 2024-09-16T15:07:32.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb027fc640 1 -- 172.21.15.133:0/1364811207 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb04017020 con 0x7fdb0c1365b0 2024-09-16T15:07:32.569 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-09-16T15:07:32.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 10 mgrc shutdown 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 -- 172.21.15.133:0/1364811207 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdaf00977c0 msgr2=0x7fdaf0099c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdaf00977c0 0x7fdaf0099c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb0801c470 tx=0x7fdb080a6000 comp rx=0 tx=0).stop 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 -- 172.21.15.133:0/1364811207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb0c1365b0 msgr2=0x7fdb0c0a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb0c1365b0 0x7fdb0c0a2ae0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c780 tx=0x7fdb0400cc40 comp rx=0 tx=0).stop 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 -- 172.21.15.133:0/1364811207 shutdown_connections 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fdaf00977c0 0x7fdaf0099c80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb0c1365b0 0x7fdb0c0a2ae0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb0c0ab810 0x7fdb0c132580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 --2- 172.21.15.133:0/1364811207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb0c0a9ae0 0x7fdb0c135160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 -- 172.21.15.133:0/1364811207 >> 172.21.15.133:0/1364811207 conn(0x7fdb0c06a030 msgr2=0x7fdb0c071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 -- 172.21.15.133:0/1364811207 shutdown_connections 2024-09-16T15:07:32.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.566+0000 7fdb13ed4640 1 -- 172.21.15.133:0/1364811207 wait complete. 2024-09-16T15:07:32.586 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-16T15:07:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b1050a640 1 Processor -- start 2024-09-16T15:07:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b1050a640 1 -- start start 2024-09-16T15:07:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b1050a640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a5960 0x7f5b0809e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b1050a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5b080a1dd0 0x7f5b080a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b1050a640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0806beb0 con 0x7f5b080a26f0 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b1050a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b080a62e0 con 0x7f5b080a1dd0 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b1050a640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0806b180 con 0x7f5b080a5960 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b0daa5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5b080a1dd0 0x7f5b080a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b0e2a6640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a5960 0x7f5b0809e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b0daa5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5b080a1dd0 0x7f5b080a21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57568/0 (socket says 172.21.15.133:57568) 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b0daa5640 1 -- 172.21.15.133:0/2024677374 learned_addr learned my addr 172.21.15.133:0/2024677374 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b0e2a6640 1 -- 172.21.15.133:0/2024677374 >> v2:172.21.15.133:3300/0 conn(0x7f5b080a1dd0 msgr2=0x7f5b080a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b0e2a6640 1 --2- 172.21.15.133:0/2024677374 >> v2:172.21.15.133:3300/0 conn(0x7f5b080a1dd0 0x7f5b080a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b0e2a6640 1 -- 172.21.15.133:0/2024677374 >> v1:172.21.15.191:6789/0 conn(0x7f5b080a26f0 legacy=0x7f5b08149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b0e2a6640 1 -- 172.21.15.133:0/2024677374 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0806b420 con 0x7f5b080a5960 2024-09-16T15:07:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.746+0000 7f5b0daa5640 1 --2- 172.21.15.133:0/2024677374 >> v2:172.21.15.133:3300/0 conn(0x7f5b080a1dd0 0x7f5b080a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:32.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b0e2a6640 1 --2- 172.21.15.133:0/2024677374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a5960 0x7f5b0809e410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5b040097c0 tx=0x7f5b0401c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8970dd3f7bd9cb61 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b0d2a4640 1 -- 172.21.15.133:0/2024677374 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b040ab600 con 0x7f5b080a5960 2024-09-16T15:07:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b0d2a4640 1 -- 172.21.15.133:0/2024677374 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b040a7050 con 0x7f5b080a5960 2024-09-16T15:07:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b0d2a4640 1 -- 172.21.15.133:0/2024677374 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b040ab980 con 0x7f5b080a5960 2024-09-16T15:07:32.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b1050a640 1 -- 172.21.15.133:0/2024677374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a5960 msgr2=0x7f5b0809e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b1050a640 1 --2- 172.21.15.133:0/2024677374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a5960 0x7f5b0809e410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5b040097c0 tx=0x7f5b0401c320 comp rx=0 tx=0).stop 2024-09-16T15:07:32.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b1050a640 1 -- 172.21.15.133:0/2024677374 shutdown_connections 2024-09-16T15:07:32.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b1050a640 1 --2- 172.21.15.133:0/2024677374 >> v2:172.21.15.133:3300/0 conn(0x7f5b080a1dd0 0x7f5b080a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b1050a640 1 --2- 172.21.15.133:0/2024677374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a5960 0x7f5b0809e410 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b1050a640 1 -- 172.21.15.133:0/2024677374 >> 172.21.15.133:0/2024677374 conn(0x7f5b080a3fd0 msgr2=0x7f5b080a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:32.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b1050a640 1 -- 172.21.15.133:0/2024677374 shutdown_connections 2024-09-16T15:07:32.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.750+0000 7f5b1050a640 1 -- 172.21.15.133:0/2024677374 wait complete. 2024-09-16T15:07:32.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b1050a640 1 Processor -- start 2024-09-16T15:07:32.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b1050a640 1 -- start start 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b1050a640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a1dd0 0x7f5b08147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b1050a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b080a26f0 0x7f5b080abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b1050a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b080a5960 0x7f5b080aa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b1050a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b08147370 con 0x7f5b080a26f0 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b1050a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b081466b0 con 0x7f5b080a5960 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b1050a640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b080a8980 con 0x7f5b080a1dd0 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0e2a6640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a1dd0 0x7f5b08147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0daa5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b080a26f0 0x7f5b080abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0e2a6640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a1dd0 0x7f5b08147960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53240/0 (socket says 172.21.15.133:53240) 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0daa5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b080a26f0 0x7f5b080abbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57576/0 (socket says 172.21.15.133:57576) 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0e2a6640 1 -- 172.21.15.133:0/4222405050 learned_addr learned my addr 172.21.15.133:0/4222405050 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0eaa7640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b080a5960 0x7f5b080aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0daa5640 1 -- 172.21.15.133:0/4222405050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a1dd0 msgr2=0x7f5b08147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0daa5640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a1dd0 0x7f5b08147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0daa5640 1 -- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b080a5960 msgr2=0x7f5b080aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0daa5640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b080a5960 0x7f5b080aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0daa5640 1 -- 172.21.15.133:0/4222405050 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af800a670 con 0x7f5b080a26f0 2024-09-16T15:07:32.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0eaa7640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b080a5960 0x7f5b080aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:07:32.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5b0daa5640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b080a26f0 0x7f5b080abbe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5af800afb0 tx=0x7f5af800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:32.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.754+0000 7f5afeffd640 1 -- 172.21.15.133:0/4222405050 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af800cc40 con 0x7f5b080a26f0 2024-09-16T15:07:32.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5afeffd640 1 -- 172.21.15.133:0/4222405050 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af800cda0 con 0x7f5b080a26f0 2024-09-16T15:07:32.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5b1050a640 1 -- 172.21.15.133:0/4222405050 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b04009420 con 0x7f5b080a26f0 2024-09-16T15:07:32.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5afeffd640 1 -- 172.21.15.133:0/4222405050 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af8004230 con 0x7f5b080a26f0 2024-09-16T15:07:32.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5b1050a640 1 -- 172.21.15.133:0/4222405050 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b08148490 con 0x7f5b080a26f0 2024-09-16T15:07:32.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5b1050a640 1 -- 172.21.15.133:0/4222405050 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b080a53d0 con 0x7f5b080a26f0 2024-09-16T15:07:32.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5afeffd640 1 -- 172.21.15.133:0/4222405050 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f5af8004390 con 0x7f5b080a26f0 2024-09-16T15:07:32.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5afeffd640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:32.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5afeffd640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:32.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5afeffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:32.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5afeffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:32.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5afeffd640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5aec097840 0x7f5aec099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:32.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5afeffd640 1 -- 172.21.15.133:0/4222405050 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f5af80c3ec0 con 0x7f5b080a26f0 2024-09-16T15:07:32.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5b0e2a6640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5aec097840 0x7f5aec099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:32.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.758+0000 7f5b0e2a6640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5aec097840 0x7f5aec099d00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b0401c360 tx=0x7f5b040a6000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:32.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.762+0000 7f5afeffd640 1 -- 172.21.15.133:0/4222405050 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5af80916d0 con 0x7f5b080a26f0 2024-09-16T15:07:32.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.906+0000 7f5b1050a640 1 -- 172.21.15.133:0/4222405050 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5b080671f0 con 0x7f5b080a26f0 2024-09-16T15:07:32.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.906+0000 7f5afeffd640 1 -- 172.21.15.133:0/4222405050 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5af8094cf0 con 0x7f5b080a26f0 2024-09-16T15:07:32.911 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 8 2024-09-16T15:07:32.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 10 mgrc shutdown 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 -- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5aec097840 msgr2=0x7f5aec099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5aec097840 0x7f5aec099d00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b0401c360 tx=0x7f5b040a6000 comp rx=0 tx=0).stop 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 -- 172.21.15.133:0/4222405050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b080a26f0 msgr2=0x7f5b080abbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b080a26f0 0x7f5b080abbe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5af800afb0 tx=0x7f5af800ecf0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 -- 172.21.15.133:0/4222405050 shutdown_connections 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b080a5960 0x7f5b080aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5aec097840 0x7f5aec099d00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b080a26f0 0x7f5b080abbe0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 --2- 172.21.15.133:0/4222405050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b080a1dd0 0x7f5b08147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 -- 172.21.15.133:0/4222405050 >> 172.21.15.133:0/4222405050 conn(0x7f5b080a3fd0 msgr2=0x7f5b080b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 -- 172.21.15.133:0/4222405050 shutdown_connections 2024-09-16T15:07:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:32.910+0000 7f5b1050a640 1 -- 172.21.15.133:0/4222405050 wait complete. 2024-09-16T15:07:32.928 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T15:07:33.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53d0219640 1 Processor -- start 2024-09-16T15:07:33.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53d0219640 1 -- start start 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53d0219640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c80a7540 0x7f53c8143590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53d0219640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f53c8143ad0 0x7f53c813f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53d0219640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53c809f490 0x7f53c813fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53d0219640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c81444e0 con 0x7f53c809f490 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53d0219640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c8143f60 con 0x7f53c8143ad0 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53d0219640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8144220 con 0x7f53c80a7540 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53ce7b6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53c809f490 0x7f53c813fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53cdfb5640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c80a7540 0x7f53c8143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53ce7b6640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f53c809f490 0x7f53c813fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57586/0 (socket says 172.21.15.133:57586) 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53ce7b6640 1 -- 172.21.15.133:0/176982192 learned_addr learned my addr 172.21.15.133:0/176982192 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53cd7b4640 1 --2- 172.21.15.133:0/176982192 >> v2:172.21.15.191:3300/0 conn(0x7f53c8143ad0 0x7f53c813f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53ce7b6640 1 -- 172.21.15.133:0/176982192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c80a7540 msgr2=0x7f53c8143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53ce7b6640 1 --2- 172.21.15.133:0/176982192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c80a7540 0x7f53c8143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53ce7b6640 1 -- 172.21.15.133:0/176982192 >> v2:172.21.15.191:3300/0 conn(0x7f53c8143ad0 msgr2=0x7f53c813f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53ce7b6640 1 --2- 172.21.15.133:0/176982192 >> v2:172.21.15.191:3300/0 conn(0x7f53c8143ad0 0x7f53c813f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.046+0000 7f53ce7b6640 1 -- 172.21.15.133:0/176982192 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80a3630 con 0x7f53c809f490 2024-09-16T15:07:33.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53ce7b6640 1 --2- 172.21.15.133:0/176982192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c809f490 0x7f53c813fcb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f53c806dbc0 tx=0x7f53c001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af13e1f0c8606371 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53ccfb3640 1 -- 172.21.15.133:0/176982192 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00a5750 con 0x7f53c809f490 2024-09-16T15:07:33.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53ccfb3640 1 -- 172.21.15.133:0/176982192 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c00a58b0 con 0x7f53c809f490 2024-09-16T15:07:33.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53ccfb3640 1 -- 172.21.15.133:0/176982192 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00a5c40 con 0x7f53c809f490 2024-09-16T15:07:33.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- 172.21.15.133:0/176982192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c809f490 msgr2=0x7f53c813fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 --2- 172.21.15.133:0/176982192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c809f490 0x7f53c813fcb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f53c806dbc0 tx=0x7f53c001c8b0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- 172.21.15.133:0/176982192 shutdown_connections 2024-09-16T15:07:33.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 --2- 172.21.15.133:0/176982192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c809f490 0x7f53c813fcb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 --2- 172.21.15.133:0/176982192 >> v2:172.21.15.191:3300/0 conn(0x7f53c8143ad0 0x7f53c813f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 --2- 172.21.15.133:0/176982192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c80a7540 0x7f53c8143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- 172.21.15.133:0/176982192 >> 172.21.15.133:0/176982192 conn(0x7f53c80a4f70 msgr2=0x7f53c80a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- 172.21.15.133:0/176982192 shutdown_connections 2024-09-16T15:07:33.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- 172.21.15.133:0/176982192 wait complete. 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 Processor -- start 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- start start 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c809f490 0x7f53c80a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c80a7540 0x7f53c81456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c81467c0 0x7f53c813cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80a16b0 con 0x7f53c81467c0 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c81471d0 con 0x7f53c80a7540 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8146f10 con 0x7f53c809f490 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cdfb5640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c809f490 0x7f53c80a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53ce7b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c81467c0 0x7f53c813cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cdfb5640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c809f490 0x7f53c80a21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53254/0 (socket says 172.21.15.133:53254) 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cdfb5640 1 -- 172.21.15.133:0/3597703624 learned_addr learned my addr 172.21.15.133:0/3597703624 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:33.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cd7b4640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c80a7540 0x7f53c81456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cdfb5640 1 -- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c80a7540 msgr2=0x7f53c81456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cdfb5640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c80a7540 0x7f53c81456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cdfb5640 1 -- 172.21.15.133:0/3597703624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c81467c0 msgr2=0x7f53c813cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cdfb5640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c81467c0 0x7f53c813cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cdfb5640 1 -- 172.21.15.133:0/3597703624 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0009b30 con 0x7f53c809f490 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cd7b4640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c80a7540 0x7f53c81456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53ce7b6640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c81467c0 0x7f53c813cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53cdfb5640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c809f490 0x7f53c80a21c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53c400e870 tx=0x7f53c400ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53be7fc640 1 -- 172.21.15.133:0/3597703624 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4009510 con 0x7f53c809f490 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53be7fc640 1 -- 172.21.15.133:0/3597703624 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c400eec0 con 0x7f53c809f490 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- 172.21.15.133:0/3597703624 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c8145fc0 con 0x7f53c809f490 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53be7fc640 1 -- 172.21.15.133:0/3597703624 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400cb40 con 0x7f53c809f490 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.050+0000 7f53d0219640 1 -- 172.21.15.133:0/3597703624 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c8146c20 con 0x7f53c809f490 2024-09-16T15:07:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.054+0000 7f53d0219640 1 -- 172.21.15.133:0/3597703624 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c8143590 con 0x7f53c809f490 2024-09-16T15:07:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.054+0000 7f53be7fc640 1 -- 172.21.15.133:0/3597703624 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f53c400cca0 con 0x7f53c809f490 2024-09-16T15:07:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.054+0000 7f53be7fc640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:33.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.054+0000 7f53be7fc640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.054+0000 7f53be7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.054+0000 7f53be7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.054+0000 7f53be7fc640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f53ac097840 0x7f53ac099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.054+0000 7f53be7fc640 1 -- 172.21.15.133:0/3597703624 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f53c40c04c0 con 0x7f53c809f490 2024-09-16T15:07:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.054+0000 7f53cd7b4640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f53ac097840 0x7f53ac099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.054+0000 7f53cd7b4640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f53ac097840 0x7f53ac099d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53c80a32a0 tx=0x7f53b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.058+0000 7f53be7fc640 1 -- 172.21.15.133:0/3597703624 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f53c408dc60 con 0x7f53c809f490 2024-09-16T15:07:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.214+0000 7f53d0219640 1 -- 172.21.15.133:0/3597703624 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53c809f060 con 0x7f53c809f490 2024-09-16T15:07:33.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.214+0000 7f53be7fc640 1 -- 172.21.15.133:0/3597703624 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f53c4091280 con 0x7f53c809f490 2024-09-16T15:07:33.220 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-09-16T15:07:33.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 10 mgrc shutdown 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 -- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f53ac097840 msgr2=0x7f53ac099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f53ac097840 0x7f53ac099d00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f53c80a32a0 tx=0x7f53b8008040 comp rx=0 tx=0).stop 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 -- 172.21.15.133:0/3597703624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c809f490 msgr2=0x7f53c80a21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c809f490 0x7f53c80a21c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53c400e870 tx=0x7f53c400ec70 comp rx=0 tx=0).stop 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 -- 172.21.15.133:0/3597703624 shutdown_connections 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f53ac097840 0x7f53ac099d00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53c81467c0 0x7f53c813cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53c80a7540 0x7f53c81456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 --2- 172.21.15.133:0/3597703624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f53c809f490 0x7f53c80a21c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 -- 172.21.15.133:0/3597703624 >> 172.21.15.133:0/3597703624 conn(0x7f53c80a4f70 msgr2=0x7f53c80a3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 -- 172.21.15.133:0/3597703624 shutdown_connections 2024-09-16T15:07:33.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.218+0000 7f53d0219640 1 -- 172.21.15.133:0/3597703624 wait complete. 2024-09-16T15:07:33.238 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T15:07:33.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f9558f80640 1 Processor -- start 2024-09-16T15:07:33.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f9558f80640 1 -- start start 2024-09-16T15:07:33.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f9558f80640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95540a3ac0 0x7f95540a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f9558f80640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9554069e60 0x7f95540a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f9558f80640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955406a430 con 0x7f955409f2a0 2024-09-16T15:07:33.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f9558f80640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95540a29b0 con 0x7f9554069e60 2024-09-16T15:07:33.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f9558f80640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95540a0340 con 0x7f95540a3ac0 2024-09-16T15:07:33.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f955259c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95540a3ac0 0x7f95540a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f955259c640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95540a3ac0 0x7f95540a3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53262/0 (socket says 172.21.15.133:53262) 2024-09-16T15:07:33.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f955259c640 1 -- 172.21.15.133:0/4013264039 learned_addr learned my addr 172.21.15.133:0/4013264039 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:33.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f9552d9d640 1 --2- 172.21.15.133:0/4013264039 >> v2:172.21.15.133:3300/0 conn(0x7f9554069e60 0x7f95540a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f955259c640 1 -- 172.21.15.133:0/4013264039 >> v2:172.21.15.133:3300/0 conn(0x7f9554069e60 msgr2=0x7f95540a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f955259c640 1 --2- 172.21.15.133:0/4013264039 >> v2:172.21.15.133:3300/0 conn(0x7f9554069e60 0x7f95540a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f955259c640 1 -- 172.21.15.133:0/4013264039 >> v1:172.21.15.191:6789/0 conn(0x7f955409f2a0 legacy=0x7f95540a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.346+0000 7f955259c640 1 -- 172.21.15.133:0/4013264039 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95540a3660 con 0x7f95540a3ac0 2024-09-16T15:07:33.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955259c640 1 --2- 172.21.15.133:0/4013264039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95540a3ac0 0x7f95540a3ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f953c009870 tx=0x7f953c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649672450ff8065e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955159a640 1 -- 172.21.15.133:0/4013264039 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0ab640 con 0x7f95540a3ac0 2024-09-16T15:07:33.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955159a640 1 -- 172.21.15.133:0/4013264039 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c0a7070 con 0x7f95540a3ac0 2024-09-16T15:07:33.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955159a640 1 -- 172.21.15.133:0/4013264039 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0ab9c0 con 0x7f95540a3ac0 2024-09-16T15:07:33.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 -- 172.21.15.133:0/4013264039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95540a3ac0 msgr2=0x7f95540a3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 --2- 172.21.15.133:0/4013264039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95540a3ac0 0x7f95540a3ea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f953c009870 tx=0x7f953c01c420 comp rx=0 tx=0).stop 2024-09-16T15:07:33.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 -- 172.21.15.133:0/4013264039 shutdown_connections 2024-09-16T15:07:33.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 --2- 172.21.15.133:0/4013264039 >> v2:172.21.15.133:3300/0 conn(0x7f9554069e60 0x7f95540a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 --2- 172.21.15.133:0/4013264039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95540a3ac0 0x7f95540a3ea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 -- 172.21.15.133:0/4013264039 >> 172.21.15.133:0/4013264039 conn(0x7f95540a4f70 msgr2=0x7f95540a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 -- 172.21.15.133:0/4013264039 shutdown_connections 2024-09-16T15:07:33.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 -- 172.21.15.133:0/4013264039 wait complete. 2024-09-16T15:07:33.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 Processor -- start 2024-09-16T15:07:33.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 -- start start 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9554069e60 0x7f95540998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f955409f2a0 0x7f9554149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95540a3ac0 0x7f9554149660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955414b030 con 0x7f9554069e60 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955413f390 con 0x7f95540a3ac0 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9558f80640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955414b230 con 0x7f955409f2a0 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9554069e60 0x7f95540998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9551d9b640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f955409f2a0 0x7f9554149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9554069e60 0x7f95540998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57610/0 (socket says 172.21.15.133:57610) 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955259c640 1 -- 172.21.15.133:0/375963254 learned_addr learned my addr 172.21.15.133:0/375963254 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f9552d9d640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95540a3ac0 0x7f9554149660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955259c640 1 -- 172.21.15.133:0/375963254 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f955409f2a0 msgr2=0x7f9554149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955259c640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f955409f2a0 0x7f9554149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955259c640 1 -- 172.21.15.133:0/375963254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95540a3ac0 msgr2=0x7f9554149660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955259c640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95540a3ac0 0x7f9554149660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.350+0000 7f955259c640 1 -- 172.21.15.133:0/375963254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c0094d0 con 0x7f9554069e60 2024-09-16T15:07:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f955259c640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9554069e60 0x7f95540998c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f953c0ad040 tx=0x7f953c01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f954affd640 1 -- 172.21.15.133:0/375963254 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0b4800 con 0x7f9554069e60 2024-09-16T15:07:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f954affd640 1 -- 172.21.15.133:0/375963254 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c0b4960 con 0x7f9554069e60 2024-09-16T15:07:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f9558f80640 1 -- 172.21.15.133:0/375963254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9554149d70 con 0x7f9554069e60 2024-09-16T15:07:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f954affd640 1 -- 172.21.15.133:0/375963254 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0b4d10 con 0x7f9554069e60 2024-09-16T15:07:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f9558f80640 1 -- 172.21.15.133:0/375963254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9554098e60 con 0x7f9554069e60 2024-09-16T15:07:33.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f9558f80640 1 -- 172.21.15.133:0/375963254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95540a2470 con 0x7f9554069e60 2024-09-16T15:07:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f954affd640 1 -- 172.21.15.133:0/375963254 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f953c0c7440 con 0x7f9554069e60 2024-09-16T15:07:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f954affd640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f954affd640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f954affd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f954affd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f954affd640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9534097840 0x7f9534099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f954affd640 1 -- 172.21.15.133:0/375963254 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f953c15a8d0 con 0x7f9554069e60 2024-09-16T15:07:33.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.354+0000 7f9551d9b640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9534097840 0x7f9534099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.358+0000 7f9551d9b640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9534097840 0x7f9534099d00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f955409e970 tx=0x7f9544009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.358+0000 7f954affd640 1 -- 172.21.15.133:0/375963254 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f953c128100 con 0x7f9554069e60 2024-09-16T15:07:33.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.506+0000 7f9558f80640 1 -- 172.21.15.133:0/375963254 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f955406b110 con 0x7f9554069e60 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f954affd640 1 -- 172.21.15.133:0/375963254 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f953c12a030 con 0x7f9554069e60 2024-09-16T15:07:33.513 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 10 mgrc shutdown 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 -- 172.21.15.133:0/375963254 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9534097840 msgr2=0x7f9534099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9534097840 0x7f9534099d00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f955409e970 tx=0x7f9544009040 comp rx=0 tx=0).stop 2024-09-16T15:07:33.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 -- 172.21.15.133:0/375963254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9554069e60 msgr2=0x7f95540998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9554069e60 0x7f95540998c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f953c0ad040 tx=0x7f953c01ac60 comp rx=0 tx=0).stop 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 -- 172.21.15.133:0/375963254 shutdown_connections 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95540a3ac0 0x7f9554149660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f955409f2a0 0x7f9554149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f9534097840 0x7f9534099d00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 --2- 172.21.15.133:0/375963254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9554069e60 0x7f95540998c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 -- 172.21.15.133:0/375963254 >> 172.21.15.133:0/375963254 conn(0x7f95540a4f70 msgr2=0x7f95540715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 -- 172.21.15.133:0/375963254 shutdown_connections 2024-09-16T15:07:33.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.510+0000 7f9558f80640 1 -- 172.21.15.133:0/375963254 wait complete. 2024-09-16T15:07:33.530 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:07:33.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.642+0000 7f296c326640 1 Processor -- start 2024-09-16T15:07:33.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.642+0000 7f296c326640 1 -- start start 2024-09-16T15:07:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296c326640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f29640a6180 0x7f29640a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296c326640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2964143ba0 0x7f2964143f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296c326640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29640a57a0 con 0x7f29640a3980 2024-09-16T15:07:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296c326640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29641438e0 con 0x7f2964143ba0 2024-09-16T15:07:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296c326640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640a3d60 con 0x7f29640a6180 2024-09-16T15:07:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296a8c3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2964143ba0 0x7f2964143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296a8c3640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2964143ba0 0x7f2964143f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57624/0 (socket says 172.21.15.133:57624) 2024-09-16T15:07:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296a8c3640 1 -- 172.21.15.133:0/3756457277 learned_addr learned my addr 172.21.15.133:0/3756457277 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f29698c1640 1 --2- 172.21.15.133:0/3756457277 >> v2:172.21.15.191:3300/0 conn(0x7f29640a6180 0x7f29640a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296a8c3640 1 -- 172.21.15.133:0/3756457277 >> v2:172.21.15.191:3300/0 conn(0x7f29640a6180 msgr2=0x7f29640a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296a8c3640 1 --2- 172.21.15.133:0/3756457277 >> v2:172.21.15.191:3300/0 conn(0x7f29640a6180 0x7f29640a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296a8c3640 1 -- 172.21.15.133:0/3756457277 >> v1:172.21.15.191:6789/0 conn(0x7f29640a3980 legacy=0x7f296406a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296a8c3640 1 -- 172.21.15.133:0/3756457277 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296406b000 con 0x7f2964143ba0 2024-09-16T15:07:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.646+0000 7f296a8c3640 1 --2- 172.21.15.133:0/3756457277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2964143ba0 0x7f2964143f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29640a7be0 tx=0x7f295401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3453106ba4352d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f29690c0640 1 -- 172.21.15.133:0/3756457277 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540a7070 con 0x7f2964143ba0 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f29690c0640 1 -- 172.21.15.133:0/3756457277 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540ab430 con 0x7f2964143ba0 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f29690c0640 1 -- 172.21.15.133:0/3756457277 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540ab7c0 con 0x7f2964143ba0 2024-09-16T15:07:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- 172.21.15.133:0/3756457277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2964143ba0 msgr2=0x7f2964143f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 --2- 172.21.15.133:0/3756457277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2964143ba0 0x7f2964143f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29640a7be0 tx=0x7f295401c3d0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- 172.21.15.133:0/3756457277 shutdown_connections 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 --2- 172.21.15.133:0/3756457277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2964143ba0 0x7f2964143f80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 --2- 172.21.15.133:0/3756457277 >> v2:172.21.15.191:3300/0 conn(0x7f29640a6180 0x7f29640a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- 172.21.15.133:0/3756457277 >> 172.21.15.133:0/3756457277 conn(0x7f29640a3fd0 msgr2=0x7f29640a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- 172.21.15.133:0/3756457277 shutdown_connections 2024-09-16T15:07:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- 172.21.15.133:0/3756457277 wait complete. 2024-09-16T15:07:33.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 Processor -- start 2024-09-16T15:07:33.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- start start 2024-09-16T15:07:33.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29640a3980 0x7f2964147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29640a6180 0x7f2964148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2964143ba0 0x7f296413bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964148850 con 0x7f29640a6180 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964149940 con 0x7f29640a3980 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296409c380 con 0x7f2964143ba0 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296a8c3640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2964143ba0 0x7f296413bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296a8c3640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2964143ba0 0x7f296413bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53274/0 (socket says 172.21.15.133:53274) 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296a8c3640 1 -- 172.21.15.133:0/3887428426 learned_addr learned my addr 172.21.15.133:0/3887428426 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f29698c1640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29640a6180 0x7f2964148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296a0c2640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29640a3980 0x7f2964147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296a8c3640 1 -- 172.21.15.133:0/3887428426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29640a3980 msgr2=0x7f2964147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296a8c3640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29640a3980 0x7f2964147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296a8c3640 1 -- 172.21.15.133:0/3887428426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29640a6180 msgr2=0x7f2964148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296a8c3640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29640a6180 0x7f2964148fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296a8c3640 1 -- 172.21.15.133:0/3887428426 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29540094d0 con 0x7f2964143ba0 2024-09-16T15:07:33.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296a8c3640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2964143ba0 0x7f296413bae0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29540ad040 tx=0x7f2954019170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f2962ffd640 1 -- 172.21.15.133:0/3887428426 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540b4830 con 0x7f2964143ba0 2024-09-16T15:07:33.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f2962ffd640 1 -- 172.21.15.133:0/3887428426 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540b4990 con 0x7f2964143ba0 2024-09-16T15:07:33.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- 172.21.15.133:0/3887428426 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2964142d30 con 0x7f2964143ba0 2024-09-16T15:07:33.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f2962ffd640 1 -- 172.21.15.133:0/3887428426 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540b4d40 con 0x7f2964143ba0 2024-09-16T15:07:33.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.650+0000 7f296c326640 1 -- 172.21.15.133:0/3887428426 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2964148190 con 0x7f2964143ba0 2024-09-16T15:07:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.654+0000 7f296c326640 1 -- 172.21.15.133:0/3887428426 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29640a6590 con 0x7f2964143ba0 2024-09-16T15:07:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.654+0000 7f2962ffd640 1 -- 172.21.15.133:0/3887428426 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f29540c2460 con 0x7f2964143ba0 2024-09-16T15:07:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.654+0000 7f2962ffd640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.654+0000 7f2962ffd640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.654+0000 7f2962ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.654+0000 7f2962ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.654+0000 7f2962ffd640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 0x7f2950099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.654+0000 7f2962ffd640 1 -- 172.21.15.133:0/3887428426 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f295415ad30 con 0x7f2964143ba0 2024-09-16T15:07:33.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.654+0000 7f296a0c2640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 0x7f2950099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.654+0000 7f296a0c2640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 0x7f2950099d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f295c005530 tx=0x7f295c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.658+0000 7f2962ffd640 1 -- 172.21.15.133:0/3887428426 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2954128440 con 0x7f2964143ba0 2024-09-16T15:07:33.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.846+0000 7f296c326640 1 -- 172.21.15.133:0/3887428426 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f296409da90 con 0x7f2964143ba0 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.846+0000 7f2962ffd640 1 -- 172.21.15.133:0/3887428426 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3978 (secure 0 0 0) 0x7f295412a370 con 0x7f2964143ba0 2024-09-16T15:07:33.851 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6811/3104644688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3104644688},{"type":"v1","addr":"172.21.15.133:6811","nonce":3104644688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6809/147821810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":147821810},{"type":"v1","addr":"172.21.15.133:6809","nonce":147821810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:07:22.048013+0000","modified":"2024-09-16T15:07:24.720576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6827/2751563681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2751563681},{"type":"v1","addr":"172.21.15.191:6827","nonce":2751563681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-16T15:07:33.852 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 10 mgrc shutdown 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 -- 172.21.15.133:0/3887428426 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 msgr2=0x7f2950099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 0x7f2950099d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f295c005530 tx=0x7f295c009040 comp rx=0 tx=0).stop 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 -- 172.21.15.133:0/3887428426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2964143ba0 msgr2=0x7f296413bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2964143ba0 0x7f296413bae0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29540ad040 tx=0x7f2954019170 comp rx=0 tx=0).stop 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 -- 172.21.15.133:0/3887428426 shutdown_connections 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2964143ba0 0x7f296413bae0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2950097840 0x7f2950099d00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29640a6180 0x7f2964148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 --2- 172.21.15.133:0/3887428426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29640a3980 0x7f2964147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 -- 172.21.15.133:0/3887428426 >> 172.21.15.133:0/3887428426 conn(0x7f29640a3fd0 msgr2=0x7f2964144f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 -- 172.21.15.133:0/3887428426 shutdown_connections 2024-09-16T15:07:33.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.850+0000 7f296c326640 1 -- 172.21.15.133:0/3887428426 wait complete. 2024-09-16T15:07:33.869 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.970+0000 7f4921321640 1 Processor -- start 2024-09-16T15:07:33.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.970+0000 7f4921321640 1 -- start start 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c0a6210 0x7f491c09edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f491c1412e0 0x7f491c14aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491c09f310 con 0x7f491c0a2fe0 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f491c0a3590 con 0x7f491c1412e0 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0a3850 con 0x7f491c0a6210 2024-09-16T15:07:33.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491affd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c0a6210 0x7f491c09edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491b7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f491c1412e0 0x7f491c14aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491affd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c0a6210 0x7f491c09edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53290/0 (socket says 172.21.15.133:53290) 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491a7fc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f491c0a2fe0 0x7f491c140bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34068/0 (socket says 172.21.15.133:34068) 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491affd640 1 -- 172.21.15.133:0/2929378480 learned_addr learned my addr 172.21.15.133:0/2929378480 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491b7fe640 1 -- 172.21.15.133:0/2929378480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c0a6210 msgr2=0x7f491c09edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491b7fe640 1 --2- 172.21.15.133:0/2929378480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c0a6210 0x7f491c09edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491b7fe640 1 -- 172.21.15.133:0/2929378480 >> v1:172.21.15.133:6789/0 conn(0x7f491c0a2fe0 legacy=0x7f491c140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4919ffb640 1 -- 172.21.15.133:0/2929378480 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582432721 0 0) 0x7f4910001120 con 0x7f491c0a2fe0 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491b7fe640 1 -- 172.21.15.133:0/2929378480 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c0a65f0 con 0x7f491c1412e0 2024-09-16T15:07:33.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491affd640 1 --2- 172.21.15.133:0/2929378480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c0a6210 0x7f491c09edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f491b7fe640 1 --2- 172.21.15.133:0/2929378480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c1412e0 0x7f491c14aca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4914009dc0 tx=0x7f491401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a7a8aa19c76d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4919ffb640 1 -- 172.21.15.133:0/2929378480 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49140a57e0 con 0x7f491c1412e0 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4919ffb640 1 -- 172.21.15.133:0/2929378480 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49140a5940 con 0x7f491c1412e0 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4919ffb640 1 -- 172.21.15.133:0/2929378480 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49140a5cd0 con 0x7f491c1412e0 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 -- 172.21.15.133:0/2929378480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c1412e0 msgr2=0x7f491c14aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 --2- 172.21.15.133:0/2929378480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c1412e0 0x7f491c14aca0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4914009dc0 tx=0x7f491401c810 comp rx=0 tx=0).stop 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 -- 172.21.15.133:0/2929378480 shutdown_connections 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 --2- 172.21.15.133:0/2929378480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c1412e0 0x7f491c14aca0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 --2- 172.21.15.133:0/2929378480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c0a6210 0x7f491c09edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 -- 172.21.15.133:0/2929378480 >> 172.21.15.133:0/2929378480 conn(0x7f491c0a3fd0 msgr2=0x7f491c0a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 -- 172.21.15.133:0/2929378480 shutdown_connections 2024-09-16T15:07:33.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.974+0000 7f4921321640 1 -- 172.21.15.133:0/2929378480 wait complete. 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 Processor -- start 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 -- start start 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c0a2fe0 0x7f491c145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f491c0a6210 0x7f491c098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c1412e0 0x7f491c09c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c09d2b0 con 0x7f491c0a2fe0 2024-09-16T15:07:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c13ffb0 con 0x7f491c0a6210 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c140500 con 0x7f491c1412e0 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f491affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c0a2fe0 0x7f491c145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f491affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c0a2fe0 0x7f491c145110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57654/0 (socket says 172.21.15.133:57654) 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f491affd640 1 -- 172.21.15.133:0/4031936769 learned_addr learned my addr 172.21.15.133:0/4031936769 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f491b7fe640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c1412e0 0x7f491c09c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f491affd640 1 -- 172.21.15.133:0/4031936769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c1412e0 msgr2=0x7f491c09c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f491affd640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c1412e0 0x7f491c09c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f491affd640 1 -- 172.21.15.133:0/4031936769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f491c0a6210 msgr2=0x7f491c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f491affd640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f491c0a6210 0x7f491c098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f491affd640 1 -- 172.21.15.133:0/4031936769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0094d0 con 0x7f491c0a2fe0 2024-09-16T15:07:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f491affd640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c0a2fe0 0x7f491c145110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f490c00e9d0 tx=0x7f490c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f49037fe640 1 -- 172.21.15.133:0/4031936769 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c00ca60 con 0x7f491c0a2fe0 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f49037fe640 1 -- 172.21.15.133:0/4031936769 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c00cbc0 con 0x7f491c0a2fe0 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f49037fe640 1 -- 172.21.15.133:0/4031936769 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c013470 con 0x7f491c0a2fe0 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 -- 172.21.15.133:0/4031936769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4914009a70 con 0x7f491c0a2fe0 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 -- 172.21.15.133:0/4031936769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c09aaa0 con 0x7f491c0a2fe0 2024-09-16T15:07:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f4921321640 1 -- 172.21.15.133:0/4031936769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c09edd0 con 0x7f491c0a2fe0 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f49037fe640 1 -- 172.21.15.133:0/4031936769 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f490c0135d0 con 0x7f491c0a2fe0 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.978+0000 7f49037fe640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.982+0000 7f49037fe640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.982+0000 7f49037fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.982+0000 7f49037fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.982+0000 7f49037fe640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4908097840 0x7f4908099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.982+0000 7f49037fe640 1 -- 172.21.15.133:0/4031936769 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f490c0c0360 con 0x7f491c0a2fe0 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.982+0000 7f491a7fc640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4908097840 0x7f4908099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.982+0000 7f491a7fc640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4908097840 0x7f4908099d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f491c140490 tx=0x7f4910001320 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:33.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:33.986+0000 7f49037fe640 1 -- 172.21.15.133:0/4031936769 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f490c08da70 con 0x7f491c0a2fe0 2024-09-16T15:07:34.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.142+0000 7f4921321640 1 -- 172.21.15.133:0/4031936769 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f491c099120 con 0x7f491c0a2fe0 2024-09-16T15:07:34.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f49037fe640 1 -- 172.21.15.133:0/4031936769 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+13441 (secure 0 0 0) 0x7f490c08f9a0 con 0x7f491c0a2fe0 2024-09-16T15:07:34.149 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":22,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:07:27.225430+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:07:20.523939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-16T15:07:21.043356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:07:34.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 10 mgrc shutdown 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 -- 172.21.15.133:0/4031936769 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4908097840 msgr2=0x7f4908099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4908097840 0x7f4908099d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f491c140490 tx=0x7f4910001320 comp rx=0 tx=0).stop 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 -- 172.21.15.133:0/4031936769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c0a2fe0 msgr2=0x7f491c145110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c0a2fe0 0x7f491c145110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f490c00e9d0 tx=0x7f490c00ee90 comp rx=0 tx=0).stop 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 -- 172.21.15.133:0/4031936769 shutdown_connections 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f491c1412e0 0x7f491c09c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f491c0a6210 0x7f491c098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f4908097840 0x7f4908099d00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 --2- 172.21.15.133:0/4031936769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f491c0a2fe0 0x7f491c145110 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 -- 172.21.15.133:0/4031936769 >> 172.21.15.133:0/4031936769 conn(0x7f491c0a3fd0 msgr2=0x7f491c143ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 -- 172.21.15.133:0/4031936769 shutdown_connections 2024-09-16T15:07:34.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.146+0000 7f4921321640 1 -- 172.21.15.133:0/4031936769 wait complete. 2024-09-16T15:07:34.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:07:34.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fbb48e640 1 Processor -- start 2024-09-16T15:07:34.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fbb48e640 1 -- start start 2024-09-16T15:07:34.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fbb48e640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 0x7f5fb40a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fbb48e640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5fb409e490 0x7f5fb406afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fbb48e640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb406a8f0 con 0x7f5fb40a1010 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fbb48e640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb40a5bd0 con 0x7f5fb409e490 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fbb48e640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40a5910 con 0x7f5fb40a2cf0 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fb922a640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 0x7f5fb40a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fb8a29640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5fb409e490 0x7f5fb406afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fb8a29640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5fb409e490 0x7f5fb406afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57660/0 (socket says 172.21.15.133:57660) 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fb9a2b640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5fb40a1010 0x7f5fb4144a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34072/0 (socket says 172.21.15.133:34072) 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fb8a29640 1 -- 172.21.15.133:0/1501526115 learned_addr learned my addr 172.21.15.133:0/1501526115 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fb922a640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 0x7f5fb40a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53314/0 (socket says 172.21.15.133:53314) 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fb8a29640 1 -- 172.21.15.133:0/1501526115 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 msgr2=0x7f5fb40a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fb8a29640 1 --2- 172.21.15.133:0/1501526115 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 0x7f5fb40a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fb8a29640 1 -- 172.21.15.133:0/1501526115 >> v1:172.21.15.133:6789/0 conn(0x7f5fb40a1010 legacy=0x7f5fb4144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.274+0000 7f5fb8a29640 1 -- 172.21.15.133:0/1501526115 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb40a4cf0 con 0x7f5fb409e490 2024-09-16T15:07:34.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb8a29640 1 --2- 172.21.15.133:0/1501526115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 0x7f5fb406afe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009870 tx=0x7f5fb001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18684d326dda4a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:34.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fabfff640 1 -- 172.21.15.133:0/1501526115 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00ab620 con 0x7f5fb409e490 2024-09-16T15:07:34.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fabfff640 1 -- 172.21.15.133:0/1501526115 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb00ab780 con 0x7f5fb409e490 2024-09-16T15:07:34.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fabfff640 1 -- 172.21.15.133:0/1501526115 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00abb10 con 0x7f5fb409e490 2024-09-16T15:07:34.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- 172.21.15.133:0/1501526115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 msgr2=0x7f5fb406afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 --2- 172.21.15.133:0/1501526115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 0x7f5fb406afe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009870 tx=0x7f5fb001c420 comp rx=0 tx=0).stop 2024-09-16T15:07:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- 172.21.15.133:0/1501526115 shutdown_connections 2024-09-16T15:07:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 --2- 172.21.15.133:0/1501526115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 0x7f5fb406afe0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 --2- 172.21.15.133:0/1501526115 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 0x7f5fb40a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- 172.21.15.133:0/1501526115 >> 172.21.15.133:0/1501526115 conn(0x7f5fb40a3fd0 msgr2=0x7f5fb40a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- 172.21.15.133:0/1501526115 shutdown_connections 2024-09-16T15:07:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- 172.21.15.133:0/1501526115 wait complete. 2024-09-16T15:07:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 Processor -- start 2024-09-16T15:07:34.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- start start 2024-09-16T15:07:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 0x7f5fb41d9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fb40a1010 0x7f5fb4146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 0x7f5fb41472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb414c010 con 0x7f5fb409e490 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb409d9a0 con 0x7f5fb40a1010 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb41da630 con 0x7f5fb40a2cf0 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb922a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 0x7f5fb41d9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb9a2b640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 0x7f5fb41472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb922a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 0x7f5fb41d9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57664/0 (socket says 172.21.15.133:57664) 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb922a640 1 -- 172.21.15.133:0/2905069276 learned_addr learned my addr 172.21.15.133:0/2905069276 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb8a29640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fb40a1010 0x7f5fb4146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb922a640 1 -- 172.21.15.133:0/2905069276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 msgr2=0x7f5fb41472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb922a640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 0x7f5fb41472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb922a640 1 -- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fb40a1010 msgr2=0x7f5fb4146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb922a640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fb40a1010 0x7f5fb4146d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb922a640 1 -- 172.21.15.133:0/2905069276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb00094d0 con 0x7f5fb409e490 2024-09-16T15:07:34.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb8a29640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fb40a1010 0x7f5fb4146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fb922a640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 0x7f5fb41d9df0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400e870 tx=0x7f5fa400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fa9ffb640 1 -- 172.21.15.133:0/2905069276 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4009510 con 0x7f5fb409e490 2024-09-16T15:07:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fa9ffb640 1 -- 172.21.15.133:0/2905069276 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa400eec0 con 0x7f5fb409e490 2024-09-16T15:07:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- 172.21.15.133:0/2905069276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb41da8f0 con 0x7f5fb409e490 2024-09-16T15:07:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fa9ffb640 1 -- 172.21.15.133:0/2905069276 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa400cb40 con 0x7f5fb409e490 2024-09-16T15:07:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.278+0000 7f5fbb48e640 1 -- 172.21.15.133:0/2905069276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb409cc20 con 0x7f5fb409e490 2024-09-16T15:07:34.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.282+0000 7f5fbb48e640 1 -- 172.21.15.133:0/2905069276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb40b3f30 con 0x7f5fb409e490 2024-09-16T15:07:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.282+0000 7f5fa9ffb640 1 -- 172.21.15.133:0/2905069276 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f5fa400cca0 con 0x7f5fb409e490 2024-09-16T15:07:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.282+0000 7f5fa9ffb640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.282+0000 7f5fa9ffb640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.282+0000 7f5fa9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.282+0000 7f5fa9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.282+0000 7f5fa9ffb640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5f98097840 0x7f5f98099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.282+0000 7f5fa9ffb640 1 -- 172.21.15.133:0/2905069276 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f5fa40c05a0 con 0x7f5fb409e490 2024-09-16T15:07:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.282+0000 7f5fb8a29640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5f98097840 0x7f5f98099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.282+0000 7f5fb8a29640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5f98097840 0x7f5f98099d00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00af000 tx=0x7f5fb0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:34.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.286+0000 7f5fa9ffb640 1 -- 172.21.15.133:0/2905069276 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5fa408dcb0 con 0x7f5fb409e490 2024-09-16T15:07:34.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 -- 172.21.15.133:0/2905069276 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fb406afe0 con 0x7f5fb409e490 2024-09-16T15:07:34.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fa9ffb640 1 -- 172.21.15.133:0/2905069276 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3978 (secure 0 0 0) 0x7f5fa4017090 con 0x7f5fb409e490 2024-09-16T15:07:34.485 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:07:34.486 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6811/3104644688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3104644688},{"type":"v1","addr":"172.21.15.133:6811","nonce":3104644688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6809/147821810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":147821810},{"type":"v1","addr":"172.21.15.133:6809","nonce":147821810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:07:22.048013+0000","modified":"2024-09-16T15:07:24.720576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6827/2751563681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2751563681},{"type":"v1","addr":"172.21.15.191:6827","nonce":2751563681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-16T15:07:34.486 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-09-16T15:07:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 10 mgrc shutdown 2024-09-16T15:07:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 -- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5f98097840 msgr2=0x7f5f98099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5f98097840 0x7f5f98099d00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00af000 tx=0x7f5fb0008040 comp rx=0 tx=0).stop 2024-09-16T15:07:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 -- 172.21.15.133:0/2905069276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 msgr2=0x7f5fb41d9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 0x7f5fb41d9df0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400e870 tx=0x7f5fa400ec70 comp rx=0 tx=0).stop 2024-09-16T15:07:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 -- 172.21.15.133:0/2905069276 shutdown_connections 2024-09-16T15:07:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5fb40a2cf0 0x7f5fb41472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fb40a1010 0x7f5fb4146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f5f98097840 0x7f5f98099d00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 --2- 172.21.15.133:0/2905069276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb409e490 0x7f5fb41d9df0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 -- 172.21.15.133:0/2905069276 >> 172.21.15.133:0/2905069276 conn(0x7f5fb40a3fd0 msgr2=0x7f5fb40b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:34.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 -- 172.21.15.133:0/2905069276 shutdown_connections 2024-09-16T15:07:34.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.482+0000 7f5fbb48e640 1 -- 172.21.15.133:0/2905069276 wait complete. 2024-09-16T15:07:34.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:07:22.048013+0000', 'modified': '2024-09-16T15:07:24.720576+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4261}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4261': {'gid': 4261, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6827/2751563681', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6826', 'nonce': 2751563681}, {'type': 'v1', 'addr': '172.21.15.191:6827', 'nonce': 2751563681}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-16T15:07:34.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:07:34.806 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-16T15:07:34.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f5841899640 1 Processor -- start 2024-09-16T15:07:34.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f5841899640 1 -- start start 2024-09-16T15:07:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f5841899640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f583c0a9ae0 0x7f583c0a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f5841899640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f583c0ab810 0x7f583c0a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f5841899640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c139b60 con 0x7f583c1365b0 2024-09-16T15:07:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f5841899640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c1399f0 con 0x7f583c0a9ae0 2024-09-16T15:07:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f5841899640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c0a5ca0 con 0x7f583c0ab810 2024-09-16T15:07:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f58327fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f583c0a9ae0 0x7f583c0a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f583affd640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f583c1365b0 0x7f583c0aa960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34078/0 (socket says 172.21.15.133:34078) 2024-09-16T15:07:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f58327fc640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f583c0a9ae0 0x7f583c0a80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57668/0 (socket says 172.21.15.133:57668) 2024-09-16T15:07:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f583affd640 1 -- 172.21.15.133:0/2305430481 learned_addr learned my addr 172.21.15.133:0/2305430481 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:07:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f583b7fe640 1 --2- 172.21.15.133:0/2305430481 >> v2:172.21.15.191:3300/0 conn(0x7f583c0ab810 0x7f583c0a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f58327fc640 1 -- 172.21.15.133:0/2305430481 >> v2:172.21.15.191:3300/0 conn(0x7f583c0ab810 msgr2=0x7f583c0a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f58327fc640 1 --2- 172.21.15.133:0/2305430481 >> v2:172.21.15.191:3300/0 conn(0x7f583c0ab810 0x7f583c0a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f58327fc640 1 -- 172.21.15.133:0/2305430481 >> v1:172.21.15.133:6789/0 conn(0x7f583c1365b0 legacy=0x7f583c0aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.930+0000 7f58327fc640 1 -- 172.21.15.133:0/2305430481 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c136990 con 0x7f583c0a9ae0 2024-09-16T15:07:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f58327fc640 1 --2- 172.21.15.133:0/2305430481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 0x7f583c0a80d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f583c0a6770 tx=0x7f582401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dca140344a85631 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f583a7fc640 1 -- 172.21.15.133:0/2305430481 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240ab600 con 0x7f583c0a9ae0 2024-09-16T15:07:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f583a7fc640 1 -- 172.21.15.133:0/2305430481 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58240a7050 con 0x7f583c0a9ae0 2024-09-16T15:07:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f583a7fc640 1 -- 172.21.15.133:0/2305430481 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240ab980 con 0x7f583c0a9ae0 2024-09-16T15:07:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f5841899640 1 -- 172.21.15.133:0/2305430481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 msgr2=0x7f583c0a80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f5841899640 1 --2- 172.21.15.133:0/2305430481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 0x7f583c0a80d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f583c0a6770 tx=0x7f582401c340 comp rx=0 tx=0).stop 2024-09-16T15:07:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f5841899640 1 -- 172.21.15.133:0/2305430481 shutdown_connections 2024-09-16T15:07:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f5841899640 1 --2- 172.21.15.133:0/2305430481 >> v2:172.21.15.191:3300/0 conn(0x7f583c0ab810 0x7f583c0a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f5841899640 1 --2- 172.21.15.133:0/2305430481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 0x7f583c0a80d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f5841899640 1 -- 172.21.15.133:0/2305430481 >> 172.21.15.133:0/2305430481 conn(0x7f583c06a030 msgr2=0x7f583c06bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f5841899640 1 -- 172.21.15.133:0/2305430481 shutdown_connections 2024-09-16T15:07:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f5841899640 1 -- 172.21.15.133:0/2305430481 wait complete. 2024-09-16T15:07:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f5841899640 1 Processor -- start 2024-09-16T15:07:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.934+0000 7f5841899640 1 -- start start 2024-09-16T15:07:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5841899640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 0x7f583c14c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5841899640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f583c0ab810 0x7f583c1dee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5841899640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c1365b0 0x7f583c1df360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5841899640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0a2ea0 con 0x7f583c0a9ae0 2024-09-16T15:07:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5841899640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c12f9d0 con 0x7f583c1365b0 2024-09-16T15:07:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5841899640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c149730 con 0x7f583c0ab810 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f583affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 0x7f583c14c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f58327fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f583c0ab810 0x7f583c1dee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f583affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 0x7f583c14c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57684/0 (socket says 172.21.15.133:57684) 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f58327fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f583c0ab810 0x7f583c1dee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53336/0 (socket says 172.21.15.133:53336) 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f583affd640 1 -- 172.21.15.133:0/1457728706 learned_addr learned my addr 172.21.15.133:0/1457728706 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f583b7fe640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c1365b0 0x7f583c1df360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f583affd640 1 -- 172.21.15.133:0/1457728706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f583c0ab810 msgr2=0x7f583c1dee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f583affd640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f583c0ab810 0x7f583c1dee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f583affd640 1 -- 172.21.15.133:0/1457728706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c1365b0 msgr2=0x7f583c1df360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f583affd640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c1365b0 0x7f583c1df360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f583affd640 1 -- 172.21.15.133:0/1457728706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5824009420 con 0x7f583c0a9ae0 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f583affd640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 0x7f583c14c310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f582c00c400 tx=0x7f582c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5838ff9640 1 -- 172.21.15.133:0/1457728706 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c005cf0 con 0x7f583c0a9ae0 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5838ff9640 1 -- 172.21.15.133:0/1457728706 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c005e50 con 0x7f583c0a9ae0 2024-09-16T15:07:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5841899640 1 -- 172.21.15.133:0/1457728706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583c1499d0 con 0x7f583c0a9ae0 2024-09-16T15:07:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5838ff9640 1 -- 172.21.15.133:0/1457728706 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0072d0 con 0x7f583c0a9ae0 2024-09-16T15:07:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5841899640 1 -- 172.21.15.133:0/1457728706 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c14bb90 con 0x7f583c0a9ae0 2024-09-16T15:07:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5838ff9640 1 -- 172.21.15.133:0/1457728706 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f582c007430 con 0x7f583c0a9ae0 2024-09-16T15:07:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5838ff9640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:07:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5838ff9640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:07:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5838ff9640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5838ff9640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:07:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.938+0000 7f5838ff9640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f581c097840 0x7f581c099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 Processor -- start 2024-09-16T15:07:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f5838ff9640 1 -- 172.21.15.133:0/1457728706 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f582c0bf1d0 con 0x7f583c0a9ae0 2024-09-16T15:07:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- start start 2024-09-16T15:07:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58327fc640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f581c097840 0x7f581c099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x565010848a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010848f70 0x5650108148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010814de0 0x5650108151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650106dcb10 con 0x565010848f70 2024-09-16T15:07:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501066baa0 con 0x565010814de0 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x565010669070 con 0x565010848650 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x565010848a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x565010848a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53352/0 (socket says 172.21.15.133:53352) 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 -- 172.21.15.133:0/3454126286 learned_addr learned my addr 172.21.15.133:0/3454126286 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58327fc640 1 --2- 172.21.15.133:0/3454126286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010848f70 0x5650108148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583b7fe640 1 --2- 172.21.15.133:0/3454126286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010814de0 0x5650108151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 -- 172.21.15.133:0/3454126286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010814de0 msgr2=0x5650108151c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 --2- 172.21.15.133:0/3454126286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010814de0 0x5650108151c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 -- 172.21.15.133:0/3454126286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010848f70 msgr2=0x5650108148a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 --2- 172.21.15.133:0/3454126286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010848f70 0x5650108148a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 -- 172.21.15.133:0/3454126286 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565010360730 con 0x565010848650 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583b7fe640 1 --2- 172.21.15.133:0/3454126286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010814de0 0x5650108151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:07:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58327fc640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f581c097840 0x7f581c099d00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58240ad000 tx=0x7f58240a6000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 --2- 172.21.15.133:0/3454126286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x565010848a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f582c00cfd0 tx=0x7f582c08c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ab55bf78691a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58317fa640 1 -- 172.21.15.133:0/3454126286 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c094460 con 0x565010848650 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58317fa640 1 -- 172.21.15.133:0/3454126286 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c098ab0 con 0x565010848650 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58317fa640 1 -- 172.21.15.133:0/3454126286 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c094160 con 0x565010848650 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- 172.21.15.133:0/3454126286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 msgr2=0x565010848a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 --2- 172.21.15.133:0/3454126286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x565010848a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f582c00cfd0 tx=0x7f582c08c990 comp rx=0 tx=0).stop 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- 172.21.15.133:0/3454126286 shutdown_connections 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 --2- 172.21.15.133:0/3454126286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010814de0 0x5650108151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 --2- 172.21.15.133:0/3454126286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010848f70 0x5650108148a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 --2- 172.21.15.133:0/3454126286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x565010848a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- 172.21.15.133:0/3454126286 >> 172.21.15.133:0/3454126286 conn(0x56501080c8d0 msgr2=0x56501081b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- 172.21.15.133:0/3454126286 shutdown_connections 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- 172.21.15.133:0/3454126286 wait complete. 2024-09-16T15:07:34.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- start start 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 Processor -- start 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010819480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x5650108199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010848f70 0x565010819f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x565010664960 con 0x565010814de0 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x565010603b70 con 0x565010848f70 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x565010621fd0 con 0x565010848650 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58449e0000 10 client.0 _ll_register_callbacks cb 0x565010800f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010819480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58327fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x5650108199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010819480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57696/0 (socket says 172.21.15.133:57696) 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 -- 172.21.15.133:0/312841852 learned_addr learned my addr 172.21.15.133:0/312841852 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58327fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x5650108199c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:53354/0 (socket says 172.21.15.133:53354) 2024-09-16T15:07:34.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583b7fe640 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010848f70 0x565010819f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58327fc640 1 -- 172.21.15.133:0/312841852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010848f70 msgr2=0x565010819f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58327fc640 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010848f70 0x565010819f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58327fc640 1 -- 172.21.15.133:0/312841852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 msgr2=0x565010819480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58327fc640 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010819480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f58327fc640 1 -- 172.21.15.133:0/312841852 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c14bb90 con 0x565010848650 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.942+0000 7f583affd640 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010819480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58327fc640 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x5650108199c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f58240ab000 tx=0x7f582401ab80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e683e09a1e27454 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f581a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3301/0 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f581a7fc640 1 -- 172.21.15.133:0/312841852 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58240c6470 con 0x565010848650 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f581a7fc640 1 -- 172.21.15.133:0/312841852 <== mon.2 v2:172.21.15.133:3301/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f58240c65d0 con 0x565010848650 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 -- 172.21.15.133:0/312841852 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565010661740 con 0x565010848650 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f581a7fc640 1 -- 172.21.15.133:0/312841852 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58240c6790 con 0x565010848650 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 10 client.4451 fetch_fsmap learned FSMap version 7 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 -- 172.21.15.133:0/312841852 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x565010629ac0 con 0x565010848650 2024-09-16T15:07:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f581a7fc640 1 -- 172.21.15.133:0/312841852 <== mon.2 v2:172.21.15.133:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565010661740 con 0x565010848650 2024-09-16T15:07:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f581a7fc640 1 -- 172.21.15.133:0/312841852 <== mon.2 v2:172.21.15.133:3301/0 5 ==== fsmap(e 7) v1 ==== 2569+0+0 (secure 0 0 0) 0x7f58240c68f0 con 0x565010848650 2024-09-16T15:07:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 10 client.4451 fetch_fsmap finished waiting for FSMap version 7 2024-09-16T15:07:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 10 client.4451 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:07:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 20 client.4451 populate_metadata read hostname 'smithi133' 2024-09-16T15:07:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.191:6826/2751563681,v1:172.21.15.191:6827/2751563681] conn(0x56501081a7d0 0x5650108590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 4 client.4451 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:07:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 -- 172.21.15.133:0/312841852 --> [v2:172.21.15.191:6826/2751563681,v1:172.21.15.191:6827/2751563681] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650106632b0 con 0x56501081a7d0 2024-09-16T15:07:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f583affd640 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.191:6826/2751563681,v1:172.21.15.191:6827/2751563681] conn(0x56501081a7d0 0x5650108590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f581a7fc640 10 client.4451 ms_handle_connect on v2:172.21.15.191:6826/2751563681 2024-09-16T15:07:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f583affd640 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.191:6826/2751563681,v1:172.21.15.191:6827/2751563681] conn(0x56501081a7d0 0x5650108590d0 crc :-1 s=READY pgs=9 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d3180ea0548ca4d server_cookie=1db696ad05e3bd7 in_seq=0 out_seq=0 2024-09-16T15:07:34.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f581a7fc640 1 -- 172.21.15.133:0/312841852 <== mds.0 v2:172.21.15.191:6826/2751563681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650106632b0 con 0x56501081a7d0 2024-09-16T15:07:34.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f581a7fc640 10 client.4451 handle_command_reply: tid=0 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 client.4451 shutdown 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 -- 172.21.15.133:0/312841852 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 msgr2=0x5650108199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x5650108199c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f58240ab000 tx=0x7f582401ab80 comp rx=0 tx=0).stop 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 -- 172.21.15.133:0/312841852 shutdown_connections 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.191:6826/2751563681,v1:172.21.15.191:6827/2751563681] conn(0x56501081a7d0 0x5650108590d0 crc :-1 s=READY pgs=9 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010848f70 0x565010819f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x5650108199c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f581a7fc640 0 client.4451 ms_handle_reset on v2:172.21.15.191:6826/2751563681 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 --2- 172.21.15.133:0/312841852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010819480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 -- 172.21.15.133:0/312841852 >> 172.21.15.133:0/312841852 conn(0x56501080c8d0 msgr2=0x565010813890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 -- 172.21.15.133:0/312841852 shutdown_connections 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 1 -- 172.21.15.133:0/312841852 wait complete. 2024-09-16T15:07:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.946+0000 7f58449e0000 20 client.4451 trim_cache size 0 max 16384 2024-09-16T15:07:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 Processor -- start 2024-09-16T15:07:34.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 -- start start 2024-09-16T15:07:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010813f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56501081a7d0 0x5650108580d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010848650 0x565010858610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x565010321940 con 0x565010814de0 2024-09-16T15:07:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x565010660150 con 0x565010848650 2024-09-16T15:07:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x565010609fd0 con 0x56501081a7d0 2024-09-16T15:07:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f583affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010813f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58327fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56501081a7d0 0x5650108580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f583affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010813f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57710/0 (socket says 172.21.15.133:57710) 2024-09-16T15:07:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f583affd640 1 -- 172.21.15.133:0/1189022436 learned_addr learned my addr 172.21.15.133:0/1189022436 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f583affd640 1 -- 172.21.15.133:0/1189022436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56501081a7d0 msgr2=0x5650108580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:34.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f583affd640 1 --2- 172.21.15.133:0/1189022436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56501081a7d0 0x5650108580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f583affd640 1 -- 172.21.15.133:0/1189022436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010848650 msgr2=0x565010858610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f583affd640 1 --2- 172.21.15.133:0/1189022436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010848650 0x565010858610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f583affd640 1 -- 172.21.15.133:0/1189022436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56501060fc10 con 0x565010814de0 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f583affd640 1 --2- 172.21.15.133:0/1189022436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010813f10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f582c090670 tx=0x7f582c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03202164589840d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f5831ffb640 1 -- 172.21.15.133:0/1189022436 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c098ab0 con 0x565010814de0 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f5831ffb640 1 -- 172.21.15.133:0/1189022436 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c08ea80 con 0x565010814de0 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f5831ffb640 1 -- 172.21.15.133:0/1189022436 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c098620 con 0x565010814de0 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 -- 172.21.15.133:0/1189022436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 msgr2=0x565010813f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 --2- 172.21.15.133:0/1189022436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010813f10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f582c090670 tx=0x7f582c01e000 comp rx=0 tx=0).stop 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 -- 172.21.15.133:0/1189022436 shutdown_connections 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 --2- 172.21.15.133:0/1189022436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x565010848650 0x565010858610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 --2- 172.21.15.133:0/1189022436 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56501081a7d0 0x5650108580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 --2- 172.21.15.133:0/1189022436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x565010813f10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 -- 172.21.15.133:0/1189022436 >> 172.21.15.133:0/1189022436 conn(0x56501080c8d0 msgr2=0x56501085b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:34.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 -- 172.21.15.133:0/1189022436 shutdown_connections 2024-09-16T15:07:34.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.962+0000 7f58449e0000 1 -- 172.21.15.133:0/1189022436 wait complete. 2024-09-16T15:07:34.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- start start 2024-09-16T15:07:34.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 Processor -- start 2024-09-16T15:07:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x56501085a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56501081a7d0 0x565010882460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x5650108829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56501060c540 con 0x565010814de0 2024-09-16T15:07:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650106674c0 con 0x56501081a7d0 2024-09-16T15:07:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x565010634250 con 0x565010848650 2024-09-16T15:07:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 10 client.0 _ll_register_callbacks cb 0x5650106ffe40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:07:34.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f583affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x56501085a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f583b7fe640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x5650108829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f583affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x56501085a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57726/0 (socket says 172.21.15.133:57726) 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f583affd640 1 -- 172.21.15.133:0/3257353552 learned_addr learned my addr 172.21.15.133:0/3257353552 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f583affd640 1 -- 172.21.15.133:0/3257353552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 msgr2=0x5650108829a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f583affd640 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x5650108829a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f583affd640 1 -- 172.21.15.133:0/3257353552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56501081a7d0 msgr2=0x565010882460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58327fc640 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56501081a7d0 0x565010882460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f583affd640 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56501081a7d0 0x565010882460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f583affd640 1 -- 172.21.15.133:0/3257353552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c0a8040 con 0x565010814de0 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f583affd640 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x56501085a870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f582c000f70 tx=0x7f582c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b918271f5d0338a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f581a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f581a7fc640 1 -- 172.21.15.133:0/3257353552 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0bc070 con 0x565010814de0 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f581a7fc640 1 -- 172.21.15.133:0/3257353552 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 4362+0+0 (secure 0 0 0) 0x7f582c098ab0 con 0x565010814de0 2024-09-16T15:07:34.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- 172.21.15.133:0/3257353552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565010634470 con 0x565010814de0 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f581a7fc640 1 -- 172.21.15.133:0/3257353552 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f582c094160 con 0x565010814de0 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 10 client.4584 fetch_fsmap learned FSMap version 7 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- 172.21.15.133:0/3257353552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x565010360b60 con 0x565010814de0 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f581a7fc640 1 -- 172.21.15.133:0/3257353552 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565010634470 con 0x565010814de0 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f581a7fc640 1 -- 172.21.15.133:0/3257353552 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 7) v1 ==== 2569+0+0 (secure 0 0 0) 0x7f582c072b40 con 0x565010814de0 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 10 client.4584 fetch_fsmap finished waiting for FSMap version 7 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 10 client.4584 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 20 client.4584 populate_metadata read hostname 'smithi133' 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.191:6826/2751563681,v1:172.21.15.191:6827/2751563681] conn(0x565010848f70 0x565010882ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 4 client.4584 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- 172.21.15.133:0/3257353552 --> [v2:172.21.15.191:6826/2751563681,v1:172.21.15.191:6827/2751563681] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5650106934a0 con 0x565010848f70 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58327fc640 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.191:6826/2751563681,v1:172.21.15.191:6827/2751563681] conn(0x565010848f70 0x565010882ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f581a7fc640 10 client.4584 ms_handle_connect on v2:172.21.15.191:6826/2751563681 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58327fc640 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.191:6826/2751563681,v1:172.21.15.191:6827/2751563681] conn(0x565010848f70 0x565010882ee0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3240bc0626ee2294 server_cookie=3ab478f45210d3b1 in_seq=0 out_seq=0 2024-09-16T15:07:34.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f581a7fc640 1 -- 172.21.15.133:0/3257353552 <== mds.0 v2:172.21.15.191:6826/2751563681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5650106934a0 con 0x565010848f70 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f581a7fc640 10 client.4584 handle_command_reply: tid=0 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 client.4584 shutdown 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- 172.21.15.133:0/3257353552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 msgr2=0x56501085a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x56501085a870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f582c000f70 tx=0x7f582c000ee0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- 172.21.15.133:0/3257353552 shutdown_connections 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.191:6826/2751563681,v1:172.21.15.191:6827/2751563681] conn(0x565010848f70 0x565010882ee0 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565010848650 0x5650108829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f581a7fc640 0 client.4584 ms_handle_reset on v2:172.21.15.191:6826/2751563681 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56501081a7d0 0x565010882460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 --2- 172.21.15.133:0/3257353552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565010814de0 0x56501085a870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- 172.21.15.133:0/3257353552 >> 172.21.15.133:0/3257353552 conn(0x56501080c8d0 msgr2=0x565010813f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- 172.21.15.133:0/3257353552 shutdown_connections 2024-09-16T15:07:34.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 1 -- 172.21.15.133:0/3257353552 wait complete. 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.966+0000 7f58449e0000 20 client.4584 trim_cache size 0 max 16384 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "9e097bec-4223-42be-ab5d-08197519f9c7", 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4261, 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 12.922092122, 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 7, 2024-09-16T15:07:34.981 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 22, 2024-09-16T15:07:34.982 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 21, 2024-09-16T15:07:34.982 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 16.921226126000001 2024-09-16T15:07:34.982 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 10 mgrc shutdown 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 -- 172.21.15.133:0/1457728706 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f581c097840 msgr2=0x7f581c099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f581c097840 0x7f581c099d00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58240ad000 tx=0x7f58240a6000 comp rx=0 tx=0).stop 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 -- 172.21.15.133:0/1457728706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 msgr2=0x7f583c14c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 0x7f583c14c310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f582c00c400 tx=0x7f582c00c8c0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 -- 172.21.15.133:0/1457728706 shutdown_connections 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c1365b0 0x7f583c1df360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f583c0ab810 0x7f583c1dee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f581c097840 0x7f581c099d00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 --2- 172.21.15.133:0/1457728706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583c0a9ae0 0x7f583c14c310 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 -- 172.21.15.133:0/1457728706 >> 172.21.15.133:0/1457728706 conn(0x7f583c06a030 msgr2=0x7f583c1381c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 -- 172.21.15.133:0/1457728706 shutdown_connections 2024-09-16T15:07:34.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:07:34.970+0000 7f5841899640 1 -- 172.21.15.133:0/1457728706 wait complete. 2024-09-16T15:07:34.988 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T15:07:35.001 INFO:tasks.workunit:Pulling workunits from ref ae7f37330c1a72493684eb1fcaed6f2a01195bb6 2024-09-16T15:07:35.001 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T15:07:35.001 DEBUG:teuthology.orchestra.run.smithi133:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T15:07:35.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:07:35.006 INFO:teuthology.orchestra.run.smithi133.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-16T15:07:35.006 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-16T15:07:35.052 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-16T15:07:35.052 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-16T15:07:35.100 INFO:tasks.workunit:timeout=3h 2024-09-16T15:07:35.100 INFO:tasks.workunit:cleanup=True 2024-09-16T15:07:35.101 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ae7f37330c1a72493684eb1fcaed6f2a01195bb6 2024-09-16T15:07:35.150 INFO:tasks.workunit.client.0.smithi133.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T15:08:02.376 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:08:02.379 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:08:32.382 INFO:tasks.workunit.client.0.smithi133.stderr:Note: switching to 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6'. 2024-09-16T15:08:32.382 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-16T15:08:32.382 INFO:tasks.workunit.client.0.smithi133.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr: git switch -c 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr:Or undo this operation with: 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr: git switch - 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T15:08:32.383 INFO:tasks.workunit.client.0.smithi133.stderr: 2024-09-16T15:08:32.384 INFO:tasks.workunit.client.0.smithi133.stderr:HEAD is now at ae7f37330c1 Merge pull request #59795 from zdover23/wip-doc-2024-09-15-backport-59613-to-reef 2024-09-16T15:08:32.390 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T15:08:32.397 INFO:tasks.workunit.client.0.smithi133.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T15:08:32.400 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T15:08:32.400 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T15:08:32.498 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:08:32.500 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:08:33.345 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T15:08:33.423 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T15:08:33.471 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T15:08:33.473 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T15:08:33.473 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T15:08:33.520 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T15:08:33.524 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-16T15:08:33.524 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T15:08:33.535 INFO:tasks.workunit:Running workunits matching mgr/test_localpool.sh on client.0... 2024-09-16T15:08:33.536 INFO:tasks.workunit:Running workunit mgr/test_localpool.sh... 2024-09-16T15:08:33.537 DEBUG:teuthology.orchestra.run.smithi133:workunit test mgr/test_localpool.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ae7f37330c1a72493684eb1fcaed6f2a01195bb6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/mgr/test_localpool.sh 2024-09-16T15:08:33.585 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph config set mgr mgr/localpool/subtree host 2024-09-16T15:08:33.698 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.692+0000 7f92a5b57640 1 Processor -- start 2024-09-16T15:08:33.698 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.692+0000 7f92a5b57640 1 -- start start 2024-09-16T15:08:33.698 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.692+0000 7f92a5b57640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92a0070210 0x7f92a0071630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:33.698 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.692+0000 7f92a5b57640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a0141850 con 0x7f92a00b1820 2024-09-16T15:08:33.698 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.692+0000 7f92a5b57640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a0141af0 con 0x7f92a0071b70 2024-09-16T15:08:33.698 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.692+0000 7f92a5b57640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a00b2e90 con 0x7f92a0070210 2024-09-16T15:08:33.699 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92a0070210 0x7f92a0071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:33.699 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929f7fe640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92a0070210 0x7f92a0071630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52496/0 (socket says 172.21.15.133:52496) 2024-09-16T15:08:33.699 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f92a00b1820 0x7f92a014bdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38458/0 (socket says 172.21.15.133:38458) 2024-09-16T15:08:33.699 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929f7fe640 1 -- 172.21.15.133:0/3083793967 learned_addr learned my addr 172.21.15.133:0/3083793967 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:33.699 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929f7fe640 1 -- 172.21.15.133:0/3083793967 >> v1:172.21.15.191:6789/0 conn(0x7f92a0071b70 legacy=0x7f92a0140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:33.699 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929f7fe640 1 -- 172.21.15.133:0/3083793967 >> v1:172.21.15.133:6789/0 conn(0x7f92a00b1820 legacy=0x7f92a014bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:33.699 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929f7fe640 1 -- 172.21.15.133:0/3083793967 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a00705f0 con 0x7f92a0070210 2024-09-16T15:08:33.701 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929f7fe640 1 --2- 172.21.15.133:0/3083793967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a0070210 0x7f92a0071630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f92900098d0 tx=0x7f929001c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc364936cf2bb0dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:33.701 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929e7fc640 1 -- 172.21.15.133:0/3083793967 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92900ab600 con 0x7f92a0070210 2024-09-16T15:08:33.701 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929e7fc640 1 -- 172.21.15.133:0/3083793967 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92900a7050 con 0x7f92a0070210 2024-09-16T15:08:33.701 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f929e7fc640 1 -- 172.21.15.133:0/3083793967 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92900ab980 con 0x7f92a0070210 2024-09-16T15:08:33.701 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f92a5b57640 1 -- 172.21.15.133:0/3083793967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a0070210 msgr2=0x7f92a0071630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:33.701 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f92a5b57640 1 --2- 172.21.15.133:0/3083793967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a0070210 0x7f92a0071630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f92900098d0 tx=0x7f929001c2e0 comp rx=0 tx=0).stop 2024-09-16T15:08:33.701 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f92a5b57640 1 -- 172.21.15.133:0/3083793967 shutdown_connections 2024-09-16T15:08:33.702 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f92a5b57640 1 --2- 172.21.15.133:0/3083793967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a0070210 0x7f92a0071630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:33.702 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f92a5b57640 1 -- 172.21.15.133:0/3083793967 >> 172.21.15.133:0/3083793967 conn(0x7f92a00b1e60 msgr2=0x7f92a00b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:33.702 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f92a5b57640 1 -- 172.21.15.133:0/3083793967 shutdown_connections 2024-09-16T15:08:33.702 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.696+0000 7f92a5b57640 1 -- 172.21.15.133:0/3083793967 wait complete. 2024-09-16T15:08:33.706 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 Processor -- start 2024-09-16T15:08:33.706 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 -- start start 2024-09-16T15:08:33.706 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0070210 0x7f92a0139280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:33.706 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92a0071b70 0x7f92a00a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:33.706 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a00b1820 0x7f92a01dd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:33.706 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a00a8d00 con 0x7f92a00b1820 2024-09-16T15:08:33.706 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a0138e40 con 0x7f92a0070210 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a01dcc70 con 0x7f92a0071b70 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929ffff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a00b1820 0x7f92a01dd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929effd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92a0071b70 0x7f92a00a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929effd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92a0071b70 0x7f92a00a84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:37598/0 (socket says 172.21.15.133:37598) 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929effd640 1 -- 172.21.15.133:0/3266169832 learned_addr learned my addr 172.21.15.133:0/3266169832 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929f7fe640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0070210 0x7f92a0139280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929ffff640 1 -- 172.21.15.133:0/3266169832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92a0071b70 msgr2=0x7f92a00a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929ffff640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92a0071b70 0x7f92a00a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929ffff640 1 -- 172.21.15.133:0/3266169832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0070210 msgr2=0x7f92a0139280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929ffff640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0070210 0x7f92a0139280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:33.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929ffff640 1 -- 172.21.15.133:0/3266169832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92900094d0 con 0x7f92a00b1820 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929effd640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92a0071b70 0x7f92a00a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f929ffff640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a00b1820 0x7f92a01dd1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f929800a110 tx=0x7f929800a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f9287fff640 1 -- 172.21.15.133:0/3266169832 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9298004d70 con 0x7f92a00b1820 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f9287fff640 1 -- 172.21.15.133:0/3266169832 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92980070d0 con 0x7f92a00b1820 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a0141620 con 0x7f92a00b1820 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f9287fff640 1 -- 172.21.15.133:0/3266169832 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92980074a0 con 0x7f92a00b1820 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a01405b0 con 0x7f92a00b1820 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a0071630 con 0x7f92a00b1820 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f9287fff640 1 -- 172.21.15.133:0/3266169832 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f9298007600 con 0x7f92a00b1820 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.700+0000 7f9287fff640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:08:33.708 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.704+0000 7f9287fff640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:08:33.709 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.704+0000 7f9287fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:33.709 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.704+0000 7f9287fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:33.709 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.704+0000 7f9287fff640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f928c097870 0x7f928c099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:33.709 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.704+0000 7f9287fff640 1 -- 172.21.15.133:0/3266169832 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4421+0+0 (secure 0 0 0) 0x7f92980bfdb0 con 0x7f92a00b1820 2024-09-16T15:08:33.709 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.704+0000 7f929f7fe640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f928c097870 0x7f928c099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:33.709 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.704+0000 7f929f7fe640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f928c097870 0x7f928c099d30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92900ad000 tx=0x7f92900a6000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:33.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.704+0000 7f9287fff640 1 -- 172.21.15.133:0/3266169832 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f929808d480 con 0x7f92a00b1820 2024-09-16T15:08:33.855 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.852+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/localpool/subtree}] v 0) v1 -- 0x7f92a006af60 con 0x7f92a00b1820 2024-09-16T15:08:33.859 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.856+0000 7f9287fff640 1 -- 172.21.15.133:0/3266169832 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/localpool/subtree}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9298018020 con 0x7f92a00b1820 2024-09-16T15:08:33.948 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.944+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/localpool/subtree}] v 0) v1 -- 0x7f92a00b6230 con 0x7f92a00b1820 2024-09-16T15:08:33.949 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.944+0000 7f9287fff640 1 -- 172.21.15.133:0/3266169832 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/localpool/subtree}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f929808f3b0 con 0x7f92a00b1820 2024-09-16T15:08:33.951 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 10 mgrc shutdown 2024-09-16T15:08:33.951 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f928c097870 msgr2=0x7f928c099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:33.951 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f928c097870 0x7f928c099d30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92900ad000 tx=0x7f92900a6000 comp rx=0 tx=0).stop 2024-09-16T15:08:33.951 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a00b1820 msgr2=0x7f92a01dd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:33.951 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a00b1820 0x7f92a01dd1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f929800a110 tx=0x7f929800a510 comp rx=0 tx=0).stop 2024-09-16T15:08:33.951 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 shutdown_connections 2024-09-16T15:08:33.952 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f928c097870 0x7f928c099d30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:33.952 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92a00b1820 0x7f92a01dd1c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:33.952 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92a0071b70 0x7f92a00a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:33.952 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 --2- 172.21.15.133:0/3266169832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92a0070210 0x7f92a0139280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:33.952 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 >> 172.21.15.133:0/3266169832 conn(0x7f92a00b1e60 msgr2=0x7f92a013bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:33.952 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 shutdown_connections 2024-09-16T15:08:33.952 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:33.948+0000 7f92a5b57640 1 -- 172.21.15.133:0/3266169832 wait complete. 2024-09-16T15:08:33.965 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph config set mgr mgr/localpool/failure_domain osd 2024-09-16T15:08:34.097 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28ec93640 1 Processor -- start 2024-09-16T15:08:34.098 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28ec93640 1 -- start start 2024-09-16T15:08:34.099 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28ec93640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff28806dda0 0x7ff28806e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.099 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28ec93640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff288070720 0x7ff288070b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.099 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28ec93640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2880b2980 con 0x7ff2880a6a60 2024-09-16T15:08:34.099 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28ec93640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2880b2c20 con 0x7ff288070720 2024-09-16T15:08:34.099 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28ec93640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2880b2ec0 con 0x7ff28806dda0 2024-09-16T15:08:34.100 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28d230640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff288070720 0x7ff288070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.100 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff287fff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff2880a6a60 0x7ff288070010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38466/0 (socket says 172.21.15.133:38466) 2024-09-16T15:08:34.100 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28d230640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff288070720 0x7ff288070b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52510/0 (socket says 172.21.15.133:52510) 2024-09-16T15:08:34.100 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff287fff640 1 -- 172.21.15.133:0/1879639264 learned_addr learned my addr 172.21.15.133:0/1879639264 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:08:34.100 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28ca2f640 1 --2- 172.21.15.133:0/1879639264 >> v2:172.21.15.191:3300/0 conn(0x7ff28806dda0 0x7ff28806e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.100 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28d230640 1 -- 172.21.15.133:0/1879639264 >> v2:172.21.15.191:3300/0 conn(0x7ff28806dda0 msgr2=0x7ff28806e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:34.100 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28d230640 1 --2- 172.21.15.133:0/1879639264 >> v2:172.21.15.191:3300/0 conn(0x7ff28806dda0 0x7ff28806e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.100 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28d230640 1 -- 172.21.15.133:0/1879639264 >> v1:172.21.15.133:6789/0 conn(0x7ff2880a6a60 legacy=0x7ff288070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:34.100 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff28d230640 1 -- 172.21.15.133:0/1879639264 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2880b1930 con 0x7ff288070720 2024-09-16T15:08:34.100 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.096+0000 7ff2877fe640 1 -- 172.21.15.133:0/1879639264 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316486626 0 0) 0x7ff2880b2980 con 0x7ff2880a6a60 2024-09-16T15:08:34.102 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28d230640 1 --2- 172.21.15.133:0/1879639264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff288070720 0x7ff288070b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff288068190 tx=0x7ff28001c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0df463361a5c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:34.102 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff2877fe640 1 -- 172.21.15.133:0/1879639264 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800a58b0 con 0x7ff288070720 2024-09-16T15:08:34.102 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff2877fe640 1 -- 172.21.15.133:0/1879639264 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2800a5a10 con 0x7ff288070720 2024-09-16T15:08:34.102 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff2877fe640 1 -- 172.21.15.133:0/1879639264 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800a5da0 con 0x7ff288070720 2024-09-16T15:08:34.103 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 -- 172.21.15.133:0/1879639264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff288070720 msgr2=0x7ff288070b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:34.103 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 --2- 172.21.15.133:0/1879639264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff288070720 0x7ff288070b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff288068190 tx=0x7ff28001c7f0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.103 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 -- 172.21.15.133:0/1879639264 shutdown_connections 2024-09-16T15:08:34.103 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 --2- 172.21.15.133:0/1879639264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff288070720 0x7ff288070b70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.103 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 --2- 172.21.15.133:0/1879639264 >> v2:172.21.15.191:3300/0 conn(0x7ff28806dda0 0x7ff28806e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.103 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 -- 172.21.15.133:0/1879639264 >> 172.21.15.133:0/1879639264 conn(0x7ff2880b1e60 msgr2=0x7ff2880b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:34.103 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 -- 172.21.15.133:0/1879639264 shutdown_connections 2024-09-16T15:08:34.104 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 -- 172.21.15.133:0/1879639264 wait complete. 2024-09-16T15:08:34.106 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 Processor -- start 2024-09-16T15:08:34.106 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 -- start start 2024-09-16T15:08:34.106 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff28806dda0 0x7ff2880ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.106 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff288070720 0x7ff2880a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.106 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2880a6a60 0x7ff2880a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.106 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880ab070 con 0x7ff2880a6a60 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880a8f00 con 0x7ff288070720 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2880ab250 con 0x7ff28806dda0 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28d230640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2880a6a60 0x7ff2880a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ca2f640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff28806dda0 0x7ff2880ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28d230640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2880a6a60 0x7ff2880a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52514/0 (socket says 172.21.15.133:52514) 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28d230640 1 -- 172.21.15.133:0/277244224 learned_addr learned my addr 172.21.15.133:0/277244224 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ca2f640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff28806dda0 0x7ff2880ab9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:37608/0 (socket says 172.21.15.133:37608) 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff287fff640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff288070720 0x7ff2880a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28d230640 1 -- 172.21.15.133:0/277244224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff28806dda0 msgr2=0x7ff2880ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28d230640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff28806dda0 0x7ff2880ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28d230640 1 -- 172.21.15.133:0/277244224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff288070720 msgr2=0x7ff2880a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:34.107 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28d230640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff288070720 0x7ff2880a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.108 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28d230640 1 -- 172.21.15.133:0/277244224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280009a70 con 0x7ff2880a6a60 2024-09-16T15:08:34.108 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ca2f640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff28806dda0 0x7ff2880ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:34.108 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28d230640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2880a6a60 0x7ff2880a3fb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff280019400 tx=0x7ff280019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:34.108 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff2857fa640 1 -- 172.21.15.133:0/277244224 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800b9860 con 0x7ff2880a6a60 2024-09-16T15:08:34.108 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff2857fa640 1 -- 172.21.15.133:0/277244224 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2800b99c0 con 0x7ff2880a6a60 2024-09-16T15:08:34.108 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2880a4fe0 con 0x7ff2880a6a60 2024-09-16T15:08:34.108 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.100+0000 7ff2857fa640 1 -- 172.21.15.133:0/277244224 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800b9d70 con 0x7ff2880a6a60 2024-09-16T15:08:34.108 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2880a81e0 con 0x7ff2880a6a60 2024-09-16T15:08:34.108 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2880a6e70 con 0x7ff2880a6a60 2024-09-16T15:08:34.109 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff2857fa640 1 -- 172.21.15.133:0/277244224 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7ff2800ad440 con 0x7ff2880a6a60 2024-09-16T15:08:34.109 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff2857fa640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:08:34.109 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff2857fa640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:08:34.109 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff2857fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:34.110 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff2857fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:34.110 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff2857fa640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff26c097840 0x7ff26c099d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.110 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff2857fa640 1 -- 172.21.15.133:0/277244224 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4421+0+0 (secure 0 0 0) 0x7ff28015b480 con 0x7ff2880a6a60 2024-09-16T15:08:34.110 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff28ca2f640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff26c097840 0x7ff26c099d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.110 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.104+0000 7ff28ca2f640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff26c097840 0x7ff26c099d00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff27c007e50 tx=0x7ff27c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:34.110 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.108+0000 7ff2857fa640 1 -- 172.21.15.133:0/277244224 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff280128be0 con 0x7ff2880a6a60 2024-09-16T15:08:34.270 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.268+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/localpool/failure_domain}] v 0) v1 -- 0x7ff2880b6230 con 0x7ff2880a6a60 2024-09-16T15:08:34.277 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.276+0000 7ff2857fa640 1 -- 172.21.15.133:0/277244224 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/localpool/failure_domain}]=0 v7)=0 v7) v1 ==== 128+0+0 (secure 0 0 0) 0x7ff28012c200 con 0x7ff2880a6a60 2024-09-16T15:08:34.374 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/localpool/failure_domain}] v 0) v1 -- 0x7ff2880ac0e0 con 0x7ff2880a6a60 2024-09-16T15:08:34.374 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff2857fa640 1 -- 172.21.15.133:0/277244224 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/localpool/failure_domain}]=0 v7)=0 v7) v1 ==== 128+0+0 (secure 0 0 0) 0x7ff2800b2070 con 0x7ff2880a6a60 2024-09-16T15:08:34.376 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 10 mgrc shutdown 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff26c097840 msgr2=0x7ff26c099d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff26c097840 0x7ff26c099d00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff27c007e50 tx=0x7ff27c008040 comp rx=0 tx=0).stop 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2880a6a60 msgr2=0x7ff2880a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2880a6a60 0x7ff2880a3fb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff280019400 tx=0x7ff280019590 comp rx=0 tx=0).stop 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 shutdown_connections 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7ff26c097840 0x7ff26c099d00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2880a6a60 0x7ff2880a3fb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff288070720 0x7ff2880a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 --2- 172.21.15.133:0/277244224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff28806dda0 0x7ff2880ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 >> 172.21.15.133:0/277244224 conn(0x7ff2880b1e60 msgr2=0x7ff2880b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 shutdown_connections 2024-09-16T15:08:34.377 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.372+0000 7ff28ec93640 1 -- 172.21.15.133:0/277244224 wait complete. 2024-09-16T15:08:34.392 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph mgr module enable localpool 2024-09-16T15:08:34.509 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.504+0000 7f300b53a640 1 Processor -- start 2024-09-16T15:08:34.509 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.504+0000 7f300b53a640 1 -- start start 2024-09-16T15:08:34.510 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f300b53a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f30040b1820 0x7f3004140b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.511 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f300b53a640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3004141850 con 0x7f3004070210 2024-09-16T15:08:34.511 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f300b53a640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3004141af0 con 0x7f3004141040 2024-09-16T15:08:34.511 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f300b53a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3004071d40 con 0x7f30040b1820 2024-09-16T15:08:34.511 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f3008ad5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f30040b1820 0x7f3004140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.511 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f30092d6640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3004070210 0x7f3004071630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38482/0 (socket says 172.21.15.133:38482) 2024-09-16T15:08:34.511 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f30092d6640 1 -- 172.21.15.133:0/3441594472 learned_addr learned my addr 172.21.15.133:0/3441594472 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:08:34.511 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f3008ad5640 1 -- 172.21.15.133:0/3441594472 >> v1:172.21.15.191:6789/0 conn(0x7f3004141040 legacy=0x7f300414bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:34.511 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f3008ad5640 1 -- 172.21.15.133:0/3441594472 >> v1:172.21.15.133:6789/0 conn(0x7f3004070210 legacy=0x7f3004071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:34.511 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f3008ad5640 1 -- 172.21.15.133:0/3441594472 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30040b2e90 con 0x7f30040b1820 2024-09-16T15:08:34.511 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f2ffbfff640 1 -- 172.21.15.133:0/3441594472 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242843755 0 0) 0x7f3004141850 con 0x7f3004070210 2024-09-16T15:08:34.513 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f3008ad5640 1 --2- 172.21.15.133:0/3441594472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30040b1820 0x7f3004140b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40097c0 tx=0x7f2ff401c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a371c494767d8545 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:34.513 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f2ffbfff640 1 -- 172.21.15.133:0/3441594472 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff40ab600 con 0x7f30040b1820 2024-09-16T15:08:34.513 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f2ffbfff640 1 -- 172.21.15.133:0/3441594472 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff40a7050 con 0x7f30040b1820 2024-09-16T15:08:34.513 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.508+0000 7f2ffbfff640 1 -- 172.21.15.133:0/3441594472 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff40ab980 con 0x7f30040b1820 2024-09-16T15:08:34.514 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- 172.21.15.133:0/3441594472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30040b1820 msgr2=0x7f3004140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:34.514 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 --2- 172.21.15.133:0/3441594472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30040b1820 0x7f3004140b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40097c0 tx=0x7f2ff401c320 comp rx=0 tx=0).stop 2024-09-16T15:08:34.514 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- 172.21.15.133:0/3441594472 shutdown_connections 2024-09-16T15:08:34.514 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 --2- 172.21.15.133:0/3441594472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30040b1820 0x7f3004140b00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.514 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- 172.21.15.133:0/3441594472 >> 172.21.15.133:0/3441594472 conn(0x7f30040b1e60 msgr2=0x7f30040b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:34.514 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- 172.21.15.133:0/3441594472 shutdown_connections 2024-09-16T15:08:34.514 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- 172.21.15.133:0/3441594472 wait complete. 2024-09-16T15:08:34.516 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 Processor -- start 2024-09-16T15:08:34.516 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- start start 2024-09-16T15:08:34.516 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3004070210 0x7f3004148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.516 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30040b1820 0x7f300413f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.516 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3004141040 0x7f300413e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.516 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3004148e20 con 0x7f3004070210 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300413fc70 con 0x7f30040b1820 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300413e050 con 0x7f3004141040 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f3009ad7640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3004141040 0x7f300413e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f30092d6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3004070210 0x7f3004148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f3009ad7640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3004141040 0x7f300413e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:37614/0 (socket says 172.21.15.133:37614) 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f3009ad7640 1 -- 172.21.15.133:0/792409906 learned_addr learned my addr 172.21.15.133:0/792409906 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f3008ad5640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30040b1820 0x7f300413f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f3009ad7640 1 -- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30040b1820 msgr2=0x7f300413f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f3009ad7640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30040b1820 0x7f300413f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f3009ad7640 1 -- 172.21.15.133:0/792409906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3004070210 msgr2=0x7f3004148610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f3009ad7640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3004070210 0x7f3004148610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f3009ad7640 1 -- 172.21.15.133:0/792409906 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4009420 con 0x7f3004141040 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f30092d6640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3004070210 0x7f3004148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f3009ad7640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3004141040 0x7f300413e880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00b440 tx=0x7f2ffc00b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:34.517 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc011830 con 0x7f3004141040 2024-09-16T15:08:34.518 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc0119b0 con 0x7f3004141040 2024-09-16T15:08:34.518 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300413e310 con 0x7f3004141040 2024-09-16T15:08:34.518 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc011d60 con 0x7f3004141040 2024-09-16T15:08:34.518 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.512+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3004147e90 con 0x7f3004141040 2024-09-16T15:08:34.518 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.516+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3004148b50 con 0x7f3004141040 2024-09-16T15:08:34.518 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.516+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 128877+0+0 (secure 0 0 0) 0x7f2ffc01a400 con 0x7f3004141040 2024-09-16T15:08:34.518 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.516+0000 7f2ff9ffb640 20 mgrc handle_mgr_map mgrmap(e 5) v1 2024-09-16T15:08:34.519 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.516+0000 7f2ff9ffb640 4 mgrc handle_mgr_map Got map version 5 2024-09-16T15:08:34.519 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.516+0000 7f2ff9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:34.519 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.516+0000 7f2ff9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:34.519 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.516+0000 7f2ff9ffb640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f2fe8099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:34.519 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.516+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4421+0+0 (secure 0 0 0) 0x7f2ffc0c0310 con 0x7f3004141040 2024-09-16T15:08:34.520 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.516+0000 7f30092d6640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f2fe8099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:34.520 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.516+0000 7f30092d6640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f2fe8099d30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3000004fd0 tx=0x7f3000001770 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:34.522 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.520+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2ffc08d9e0 con 0x7f3004141040 2024-09-16T15:08:34.712 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:34.708+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "mgr module enable", "module": "localpool"} v 0) v1 -- 0x7f3004140b00 con 0x7f3004141040 2024-09-16T15:08:35.295 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.292+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mgrmap(e 6) v1 ==== 128890+0+0 (secure 0 0 0) 0x7f2ffc08ddb0 con 0x7f3004141040 2024-09-16T15:08:35.295 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.292+0000 7f2ff9ffb640 20 mgrc handle_mgr_map mgrmap(e 6) v1 2024-09-16T15:08:35.295 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.292+0000 7f2ff9ffb640 4 mgrc handle_mgr_map Got map version 6 2024-09-16T15:08:35.295 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.292+0000 7f2ff9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:35.296 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.292+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 <== mon.2 v2:172.21.15.133:3301/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "localpool"}]=0 v6) v1 ==== 88+0+0 (secure 0 0 0) 0x7f2ffc085420 con 0x7f3004141040 2024-09-16T15:08:35.318 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.316+0000 7f30092d6640 1 -- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 msgr2=0x7f2fe8099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-16T15:08:35.318 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.316+0000 7f30092d6640 1 -- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 msgr2=0x7f2fe8099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:08:35.318 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.316+0000 7f30092d6640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f2fe8099d30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3000004fd0 tx=0x7f3000001770 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:08:35.318 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.316+0000 7f30092d6640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f2fe8099d30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3000004fd0 tx=0x7f3000001770 comp rx=0 tx=0).stop 2024-09-16T15:08:35.318 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.316+0000 7f2ff9ffb640 4 mgrc ms_handle_reset ms_handle_reset con 0x7f2fe8097870 2024-09-16T15:08:35.318 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.316+0000 7f2ff9ffb640 4 mgrc reconnect Terminating session with v2:172.21.15.191:6824/3926506186 2024-09-16T15:08:35.318 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.316+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 msgr2=0x7f2fe8099d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:08:35.318 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.316+0000 7f2ff9ffb640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f2fe8099d30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:35.318 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.316+0000 7f2ff9ffb640 4 mgrc reconnect waiting to retry connect until 709.321702s 2024-09-16T15:08:35.414 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:35.412+0000 7f3a55a8e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T15:08:35.419 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:35.415+0000 7fa518c13180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T15:08:35.419 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:35.415+0000 7f76f4142180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T15:08:35.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.432+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "mgr module enable", "module": "localpool"} v 0) v1 -- 0x7f2fdc005070 con 0x7f3004141040 2024-09-16T15:08:35.520 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.516+0000 7f2feffff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:35.520 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.516+0000 7f2feffff640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f300413edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:35.520 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.516+0000 7f30092d6640 1 -- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 msgr2=0x7f300413edc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/3926506186 2024-09-16T15:08:35.520 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.516+0000 7f30092d6640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f300413edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:08:35.722 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.720+0000 7f30092d6640 1 -- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 msgr2=0x7f300413edc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/3926506186 2024-09-16T15:08:35.722 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:35.720+0000 7f30092d6640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f300413edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:08:35.773 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:35.772+0000 7f3a55a8e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T15:08:35.802 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:35.799+0000 7f76f4142180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T15:08:35.807 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:35.803+0000 7fa518c13180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T15:08:35.900 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:35.896+0000 7f3a55a8e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T15:08:35.936 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:35.931+0000 7f76f4142180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T15:08:35.940 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:35.935+0000 7fa518c13180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T15:08:36.126 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.124+0000 7f30092d6640 1 -- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 msgr2=0x7f300413edc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/3926506186 2024-09-16T15:08:36.126 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.124+0000 7f30092d6640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f300413edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:08:36.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.292+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 <== mon.2 v2:172.21.15.133:3301/0 9 ==== mgrmap(e 7) v1 ==== 128890+0+0 (secure 0 0 0) 0x7f2ffc0bed90 con 0x7f3004141040 2024-09-16T15:08:36.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.292+0000 7f2ff9ffb640 20 mgrc handle_mgr_map mgrmap(e 7) v1 2024-09-16T15:08:36.298 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.296+0000 7f2ff9ffb640 4 mgrc handle_mgr_map Got map version 7 2024-09-16T15:08:36.298 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.296+0000 7f2ff9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:36.299 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.296+0000 7f2ff9ffb640 1 -- 172.21.15.133:0/792409906 <== mon.2 v2:172.21.15.133:3301/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "localpool"}]=0 module 'localpool' is already enabled v7) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2ffc0c50b0 con 0x7f3004141040 2024-09-16T15:08:36.299 INFO:tasks.workunit.client.0.smithi133.stderr:module 'localpool' is already enabled 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.296+0000 7f300b53a640 10 mgrc shutdown 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.296+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 msgr2=0x7f300413edc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.296+0000 7f300b53a640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f300413edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.296+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3004141040 msgr2=0x7f300413e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.296+0000 7f300b53a640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3004141040 0x7f300413e880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00b440 tx=0x7f2ffc00b900 comp rx=0 tx=0).stop 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.300+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 shutdown_connections 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.300+0000 7f300b53a640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3004141040 0x7f300413e880 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.300+0000 7f300b53a640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30040b1820 0x7f300413f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.300+0000 7f300b53a640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7f2fe8097870 0x7f300413edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.300+0000 7f300b53a640 1 --2- 172.21.15.133:0/792409906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3004070210 0x7f3004148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.300+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 >> 172.21.15.133:0/792409906 conn(0x7f30040b1e60 msgr2=0x7f30040b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:36.303 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.300+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 shutdown_connections 2024-09-16T15:08:36.303 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.300+0000 7f300b53a640 1 -- 172.21.15.133:0/792409906 wait complete. 2024-09-16T15:08:36.316 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:36.312+0000 7f3a55a8e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:36.319 INFO:tasks.workunit.client.0.smithi133.stderr:+ + grep ^by-host- 2024-09-16T15:08:36.320 INFO:tasks.workunit.client.0.smithi133.stderr:ceph osd pool ls 2024-09-16T15:08:36.360 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:36.355+0000 7f76f4142180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:36.363 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:36.359+0000 7fa518c13180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:36.368 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:36.364+0000 7f3a55a8e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T15:08:36.412 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:36.407+0000 7f76f4142180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T15:08:36.415 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:36.411+0000 7fa518c13180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T15:08:36.418 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:36.416+0000 7f3a55a8e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T15:08:36.434 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b8b4a640 1 Processor -- start 2024-09-16T15:08:36.434 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b8b4a640 1 -- start start 2024-09-16T15:08:36.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b8b4a640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8b40abc90 0x7fd8b4071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:36.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b8b4a640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd8b406a5d0 0x7fd8b414e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:36.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b8b4a640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b40a9110 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b8b4a640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b40a8b10 con 0x7fd8b40abc90 2024-09-16T15:08:36.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b8b4a640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b40a8c80 con 0x7fd8b406a5d0 2024-09-16T15:08:36.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8b40abc90 0x7fd8b4071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:36.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b259c640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd8b40abc90 0x7fd8b4071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52560/0 (socket says 172.21.15.133:52560) 2024-09-16T15:08:36.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b259c640 1 -- 172.21.15.133:0/456415365 learned_addr learned my addr 172.21.15.133:0/456415365 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:36.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b2d9d640 1 --2- 172.21.15.133:0/456415365 >> v2:172.21.15.191:3300/0 conn(0x7fd8b406a5d0 0x7fd8b414e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:36.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b259c640 1 -- 172.21.15.133:0/456415365 >> v2:172.21.15.191:3300/0 conn(0x7fd8b406a5d0 msgr2=0x7fd8b414e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:36.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b259c640 1 --2- 172.21.15.133:0/456415365 >> v2:172.21.15.191:3300/0 conn(0x7fd8b406a5d0 0x7fd8b414e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b259c640 1 -- 172.21.15.133:0/456415365 >> v1:172.21.15.191:6789/0 conn(0x7fd8b40b5ae0 legacy=0x7fd8b4069ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:36.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b259c640 1 -- 172.21.15.133:0/456415365 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b40aa660 con 0x7fd8b40abc90 2024-09-16T15:08:36.437 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.432+0000 7fd8b259c640 1 --2- 172.21.15.133:0/456415365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40abc90 0x7fd8b4071610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40a6fd0 tx=0x7fd8a401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44064b70e2987fb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:36.437 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b159a640 1 -- 172.21.15.133:0/456415365 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40ab600 con 0x7fd8b40abc90 2024-09-16T15:08:36.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b159a640 1 -- 172.21.15.133:0/456415365 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a40a7050 con 0x7fd8b40abc90 2024-09-16T15:08:36.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b159a640 1 -- 172.21.15.133:0/456415365 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40ab980 con 0x7fd8b40abc90 2024-09-16T15:08:36.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/456415365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40abc90 msgr2=0x7fd8b4071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:36.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 --2- 172.21.15.133:0/456415365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40abc90 0x7fd8b4071610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40a6fd0 tx=0x7fd8a401c430 comp rx=0 tx=0).stop 2024-09-16T15:08:36.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/456415365 shutdown_connections 2024-09-16T15:08:36.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 --2- 172.21.15.133:0/456415365 >> v2:172.21.15.191:3300/0 conn(0x7fd8b406a5d0 0x7fd8b414e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 --2- 172.21.15.133:0/456415365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40abc90 0x7fd8b4071610 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/456415365 >> 172.21.15.133:0/456415365 conn(0x7fd8b4073150 msgr2=0x7fd8b4073560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:36.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/456415365 shutdown_connections 2024-09-16T15:08:36.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/456415365 wait complete. 2024-09-16T15:08:36.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 Processor -- start 2024-09-16T15:08:36.440 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- start start 2024-09-16T15:08:36.441 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b406a5d0 0x7fd8b4141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:36.441 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8b40abc90 0x7fd8b41482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:36.441 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40b5ae0 0x7fd8b41df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:36.441 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4147630 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40b4140 con 0x7fd8b406a5d0 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b41df330 con 0x7fd8b40abc90 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40b5ae0 0x7fd8b41df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b1d9b640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8b40abc90 0x7fd8b41482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b2d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40b5ae0 0x7fd8b41df8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52562/0 (socket says 172.21.15.133:52562) 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b2d9d640 1 -- 172.21.15.133:0/2941055028 learned_addr learned my addr 172.21.15.133:0/2941055028 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b259c640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b406a5d0 0x7fd8b4141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b2d9d640 1 -- 172.21.15.133:0/2941055028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8b40abc90 msgr2=0x7fd8b41482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b2d9d640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8b40abc90 0x7fd8b41482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b2d9d640 1 -- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b406a5d0 msgr2=0x7fd8b4141b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b2d9d640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b406a5d0 0x7fd8b4141b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b2d9d640 1 -- 172.21.15.133:0/2941055028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a40094d0 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b1d9b640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8b40abc90 0x7fd8b41482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:36.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b2d9d640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40b5ae0 0x7fd8b41df8a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac007e40 tx=0x7fd8ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:36.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8aaffd640 1 -- 172.21.15.133:0/2941055028 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac014620 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8aaffd640 1 -- 172.21.15.133:0/2941055028 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac00ee90 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b41df5f0 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8aaffd640 1 -- 172.21.15.133:0/2941055028 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0149e0 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.436+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b4147bd0 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.440+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b4071610 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.440+0000 7fd8aaffd640 1 -- 172.21.15.133:0/2941055028 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 7) v1 ==== 128890+0+0 (secure 0 0 0) 0x7fd8ac014b40 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.440+0000 7fd8aaffd640 20 mgrc handle_mgr_map mgrmap(e 7) v1 2024-09-16T15:08:36.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.440+0000 7fd8aaffd640 4 mgrc handle_mgr_map Got map version 7 2024-09-16T15:08:36.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.440+0000 7fd8aaffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:36.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.440+0000 7fd8aaffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] 2024-09-16T15:08:36.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.440+0000 7fd8aaffd640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd894097890 0x7fd894099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:36.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.440+0000 7fd8b259c640 1 -- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd894097890 msgr2=0x7fd894099d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/3926506186 2024-09-16T15:08:36.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.440+0000 7fd8aaffd640 1 -- 172.21.15.133:0/2941055028 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4421+0+0 (secure 0 0 0) 0x7fd8ac0c0670 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.440+0000 7fd8b259c640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd894097890 0x7fd894099d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:08:36.447 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.444+0000 7fd8aaffd640 1 -- 172.21.15.133:0/2941055028 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd8ac08dd40 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.462 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:36.459+0000 7f76f4142180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T15:08:36.464 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:36.463+0000 7fa518c13180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T15:08:36.512 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:36.508+0000 7f3a55a8e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T15:08:36.538 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:36.535+0000 7f76f4142180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T15:08:36.540 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:36.535+0000 7fa518c13180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T15:08:36.564 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:36.560+0000 7f3a55a8e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T15:08:36.588 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:36.587+0000 7f76f4142180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T15:08:36.591 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:36.587+0000 7fa518c13180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T15:08:36.613 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.612+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7fd8b40a92b0 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.614 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.612+0000 7fd8aaffd640 1 -- 172.21.15.133:0/2941055028 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v24) v1 ==== 59+0+37 (secure 0 0 0) 0x7fd8ac08fc70 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.639 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:36.636+0000 7f3a55a8e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T15:08:36.645 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.644+0000 7fd8b259c640 1 -- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd894097890 msgr2=0x7fd894099d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/3926506186 2024-09-16T15:08:36.645 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.644+0000 7fd8b259c640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd894097890 0x7fd894099d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:08:36.663 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:36.659+0000 7f76f4142180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T15:08:36.665 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:36.663+0000 7fa518c13180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T15:08:36.688 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:36.684+0000 7f3a55a8e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T15:08:36.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.704+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7fd8b414e060 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.707 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.704+0000 7fd8aaffd640 1 -- 172.21.15.133:0/2941055028 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v24) v1 ==== 59+0+37 (secure 0 0 0) 0x7fd8ac0913f0 con 0x7fd8b40b5ae0 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.704+0000 7fd8b8b4a640 10 mgrc shutdown 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd894097890 msgr2=0x7fd894099d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd894097890 0x7fd894099d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40b5ae0 msgr2=0x7fd8b41df8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40b5ae0 0x7fd8b41df8a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac007e40 tx=0x7fd8ac00e4b0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 shutdown_connections 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:6824/3926506186,v1:172.21.15.191:6825/3926506186] conn(0x7fd894097890 0x7fd894099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8b40b5ae0 0x7fd8b41df8a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8b40abc90 0x7fd8b41482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 --2- 172.21.15.133:0/2941055028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b406a5d0 0x7fd8b4141b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:36.710 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 >> 172.21.15.133:0/2941055028 conn(0x7fd8b4073150 msgr2=0x7fd8b40b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:36.711 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 shutdown_connections 2024-09-16T15:08:36.711 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:36.708+0000 7fd8b8b4a640 1 -- 172.21.15.133:0/2941055028 wait complete. 2024-09-16T15:08:36.712 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:36.707+0000 7f76f4142180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T15:08:36.713 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:36.711+0000 7fa518c13180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T15:08:36.724 INFO:tasks.workunit.client.0.smithi133.stderr:+ sleep 5 2024-09-16T15:08:36.830 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:36.828+0000 7f3a55a8e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T15:08:36.853 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:36.851+0000 7f76f4142180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T15:08:36.855 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:36.851+0000 7fa518c13180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T15:08:36.881 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:36.876+0000 7f3a55a8e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T15:08:36.904 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:36.903+0000 7f76f4142180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T15:08:36.906 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:36.903+0000 7fa518c13180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T15:08:37.096 INFO:tasks.ceph.mgr.x.smithi133.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T15:08:37.097 INFO:tasks.ceph.mgr.x.smithi133.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T15:08:37.097 INFO:tasks.ceph.mgr.x.smithi133.stderr: from numpy import show_config as show_numpy_config 2024-09-16T15:08:37.104 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:37.100+0000 7f3a55a8e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T15:08:37.123 INFO:tasks.ceph.mgr.y.smithi191.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T15:08:37.123 INFO:tasks.ceph.mgr.y.smithi191.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T15:08:37.123 INFO:tasks.ceph.mgr.y.smithi191.stderr: from numpy import show_config as show_numpy_config 2024-09-16T15:08:37.124 INFO:tasks.ceph.mgr.z.smithi191.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T15:08:37.124 INFO:tasks.ceph.mgr.z.smithi191.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T15:08:37.124 INFO:tasks.ceph.mgr.z.smithi191.stderr: from numpy import show_config as show_numpy_config 2024-09-16T15:08:37.131 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:37.127+0000 7fa518c13180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T15:08:37.132 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:37.127+0000 7f76f4142180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T15:08:37.234 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:37.228+0000 7f3a55a8e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:37.263 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:37.259+0000 7fa518c13180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:37.264 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:37.259+0000 7f76f4142180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:37.342 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:37.340+0000 7f3a55a8e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:37.373 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:37.371+0000 7fa518c13180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:37.375 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:37.371+0000 7f76f4142180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:37.402 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:37.400+0000 7f3a55a8e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T15:08:37.435 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:37.431+0000 7fa518c13180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T15:08:37.437 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:37.435+0000 7f76f4142180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T15:08:37.533 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:37.528+0000 7f3a55a8e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:37.569 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:37.567+0000 7fa518c13180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:37.570 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:37.567+0000 7f76f4142180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:38.000 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:37.996+0000 7f3a55a8e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T15:08:38.051 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:38.047+0000 7fa518c13180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T15:08:38.052 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.047+0000 7f76f4142180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T15:08:38.072 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:38.068+0000 7f3a55a8e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T15:08:38.126 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:38.123+0000 7fa518c13180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T15:08:38.126 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.123+0000 7f76f4142180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T15:08:38.212 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:38.208+0000 7f3a55a8e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T15:08:38.268 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:38.263+0000 7fa518c13180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T15:08:38.269 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.263+0000 7f76f4142180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T15:08:38.282 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:38.280+0000 7f3a55a8e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T15:08:38.343 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.339+0000 7f76f4142180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T15:08:38.343 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:38.339+0000 7fa518c13180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T15:08:38.560 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:38.556+0000 7f3a55a8e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T15:08:38.613 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:38.608+0000 7f3a55a8e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T15:08:38.624 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:38.619+0000 7fa518c13180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T15:08:38.624 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.619+0000 7f76f4142180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T15:08:38.678 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:38.675+0000 7fa518c13180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T15:08:38.678 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.675+0000 7f76f4142180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T15:08:38.772 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.767+0000 7f767f6ba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:38.772 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.767+0000 7f767f6ba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:38.772 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.767+0000 7f767f6ba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:38.772 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.767+0000 7f767f6ba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:38.772 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.767+0000 7f767f6ba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:38.772 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.767+0000 7f767c6b4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:38.772 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.767+0000 7f767c6b4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:38.772 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.767+0000 7f767c6b4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:38.773 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.767+0000 7f767c6b4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:38.773 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:38.767+0000 7f767c6b4640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:41.726 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph osd pool ls 2024-09-16T15:08:41.726 INFO:tasks.workunit.client.0.smithi133.stderr:+ grep ^by-host- 2024-09-16T15:08:41.864 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7f0695640 1 Processor -- start 2024-09-16T15:08:41.864 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7f0695640 1 -- start start 2024-09-16T15:08:41.865 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7f0695640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7e80aba60 0x7fb7e806aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7f0695640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8070220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7f0695640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e806b870 con 0x7fb7e80ab320 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7f0695640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e80b55b0 con 0x7fb7e80aba60 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7f0695640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80acd60 con 0x7fb7e806b410 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7ee431640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7e80aba60 0x7fb7e806aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7ee431640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7e80aba60 0x7fb7e806aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49818/0 (socket says 172.21.15.133:49818) 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7ee431640 1 -- 172.21.15.133:0/3759052399 learned_addr learned my addr 172.21.15.133:0/3759052399 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7eec32640 1 --2- 172.21.15.133:0/3759052399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7ee431640 1 -- 172.21.15.133:0/3759052399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 msgr2=0x7fb7e8070220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7ee431640 1 --2- 172.21.15.133:0/3759052399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8070220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:41.866 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7ee431640 1 -- 172.21.15.133:0/3759052399 >> v1:172.21.15.191:6789/0 conn(0x7fb7e80ab320 legacy=0x7fb7e806fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:41.867 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7ee431640 1 -- 172.21.15.133:0/3759052399 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e80acfe0 con 0x7fb7e80aba60 2024-09-16T15:08:41.867 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.860+0000 7fb7eec32640 1 --2- 172.21.15.133:0/3759052399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:41.867 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7ee431640 1 --2- 172.21.15.133:0/3759052399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e80aba60 0x7fb7e806aed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009870 tx=0x7fb7e401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769fe1f91adb7ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:41.867 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7ed42f640 1 -- 172.21.15.133:0/3759052399 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40ab650 con 0x7fb7e80aba60 2024-09-16T15:08:41.868 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7ed42f640 1 -- 172.21.15.133:0/3759052399 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e40ab7b0 con 0x7fb7e80aba60 2024-09-16T15:08:41.868 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7ed42f640 1 -- 172.21.15.133:0/3759052399 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40abb40 con 0x7fb7e80aba60 2024-09-16T15:08:41.868 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7f0695640 1 -- 172.21.15.133:0/3759052399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e80aba60 msgr2=0x7fb7e806aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:41.869 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7f0695640 1 --2- 172.21.15.133:0/3759052399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e80aba60 0x7fb7e806aed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009870 tx=0x7fb7e401c470 comp rx=0 tx=0).stop 2024-09-16T15:08:41.869 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7f0695640 1 -- 172.21.15.133:0/3759052399 shutdown_connections 2024-09-16T15:08:41.869 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7f0695640 1 --2- 172.21.15.133:0/3759052399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:41.869 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7f0695640 1 --2- 172.21.15.133:0/3759052399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e80aba60 0x7fb7e806aed0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:41.869 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7f0695640 1 -- 172.21.15.133:0/3759052399 >> 172.21.15.133:0/3759052399 conn(0x7fb7e80b5720 msgr2=0x7fb7e80b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:41.869 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7f0695640 1 -- 172.21.15.133:0/3759052399 shutdown_connections 2024-09-16T15:08:41.869 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.864+0000 7fb7f0695640 1 -- 172.21.15.133:0/3759052399 wait complete. 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 Processor -- start 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 -- start start 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7e80ab320 0x7fb7e8133280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e80aba60 0x7fb7e81337c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e81341d0 con 0x7fb7e80aba60 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8133d00 con 0x7fb7e80ab320 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8149e70 con 0x7fb7e806b410 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7ee431640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7eec32640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e80aba60 0x7fb7e81337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:41.872 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7ee431640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:42330/0 (socket says 172.21.15.133:42330) 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7ee431640 1 -- 172.21.15.133:0/3351740233 learned_addr learned my addr 172.21.15.133:0/3351740233 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7edc30640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7e80ab320 0x7fb7e8133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7eec32640 1 -- 172.21.15.133:0/3351740233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 msgr2=0x7fb7e8149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7eec32640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7eec32640 1 -- 172.21.15.133:0/3351740233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7e80ab320 msgr2=0x7fb7e8133280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7eec32640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7e80ab320 0x7fb7e8133280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7eec32640 1 -- 172.21.15.133:0/3351740233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0009a70 con 0x7fb7e80aba60 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7ee431640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7eec32640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e80aba60 0x7fb7e81337c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009e50 tx=0x7fb7e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7deffd640 1 -- 172.21.15.133:0/3351740233 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000cd30 con 0x7fb7e80aba60 2024-09-16T15:08:41.873 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7deffd640 1 -- 172.21.15.133:0/3351740233 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e000f040 con 0x7fb7e80aba60 2024-09-16T15:08:41.874 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e40094d0 con 0x7fb7e80aba60 2024-09-16T15:08:41.874 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7deffd640 1 -- 172.21.15.133:0/3351740233 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0003be0 con 0x7fb7e80aba60 2024-09-16T15:08:41.874 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e8133f10 con 0x7fb7e80aba60 2024-09-16T15:08:41.874 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e8149960 con 0x7fb7e80aba60 2024-09-16T15:08:41.875 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7deffd640 1 -- 172.21.15.133:0/3351740233 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 10) v1 ==== 128936+0+0 (secure 0 0 0) 0x7fb7e0014480 con 0x7fb7e80aba60 2024-09-16T15:08:41.875 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.868+0000 7fb7deffd640 20 mgrc handle_mgr_map mgrmap(e 10) v1 2024-09-16T15:08:41.875 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.872+0000 7fb7deffd640 4 mgrc handle_mgr_map Got map version 10 2024-09-16T15:08:41.875 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.872+0000 7fb7deffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:41.875 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.872+0000 7fb7deffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:41.875 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.872+0000 7fb7deffd640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fb7cc097920 0x7fb7cc099de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:41.875 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.872+0000 7fb7deffd640 1 -- 172.21.15.133:0/3351740233 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5108+0+0 (secure 0 0 0) 0x7fb7e00c0bd0 con 0x7fb7e80aba60 2024-09-16T15:08:41.875 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.872+0000 7fb7ee431640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fb7cc097920 0x7fb7cc099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:41.875 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.872+0000 7fb7ee431640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fb7cc097920 0x7fb7cc099de0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb7e401c4b0 tx=0x7fb7e40b6040 comp rx=0 tx=0).ready entity=mgr.4632 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:41.878 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:41.876+0000 7fb7deffd640 1 -- 172.21.15.133:0/3351740233 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb7e8149960 con 0x7fb7e80aba60 2024-09-16T15:08:42.036 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.032+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7fb7e806aed0 con 0x7fb7e80aba60 2024-09-16T15:08:42.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.032+0000 7fb7deffd640 1 -- 172.21.15.133:0/3351740233 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v28) v1 ==== 59+0+55 (secure 0 0 0) 0x7fb7e007ce60 con 0x7fb7e80aba60 2024-09-16T15:08:42.131 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.128+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7fb7e80ad870 con 0x7fb7e80aba60 2024-09-16T15:08:42.131 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.128+0000 7fb7deffd640 1 -- 172.21.15.133:0/3351740233 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v28) v1 ==== 59+0+55 (secure 0 0 0) 0x7fb7e0018020 con 0x7fb7e80aba60 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.128+0000 7fb7f0695640 10 mgrc shutdown 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.128+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fb7cc097920 msgr2=0x7fb7cc099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.128+0000 7fb7f0695640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fb7cc097920 0x7fb7cc099de0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb7e401c4b0 tx=0x7fb7e40b6040 comp rx=0 tx=0).stop 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.132+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e80aba60 msgr2=0x7fb7e81337c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.132+0000 7fb7f0695640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e80aba60 0x7fb7e81337c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009e50 tx=0x7fb7e000c490 comp rx=0 tx=0).stop 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.132+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 shutdown_connections 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.132+0000 7fb7f0695640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fb7cc097920 0x7fb7cc099de0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.132+0000 7fb7f0695640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7e80aba60 0x7fb7e81337c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.132+0000 7fb7f0695640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7e80ab320 0x7fb7e8133280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.132+0000 7fb7f0695640 1 --2- 172.21.15.133:0/3351740233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7e806b410 0x7fb7e8149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.132+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 >> 172.21.15.133:0/3351740233 conn(0x7fb7e80b5720 msgr2=0x7fb7e806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:42.134 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.132+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 shutdown_connections 2024-09-16T15:08:42.135 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.132+0000 7fb7f0695640 1 -- 172.21.15.133:0/3351740233 wait complete. 2024-09-16T15:08:42.148 INFO:tasks.workunit.client.0.smithi133.stdout:by-host-smithi133 2024-09-16T15:08:42.149 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph mgr module disable localpool 2024-09-16T15:08:42.265 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.260+0000 7fad084a4640 1 Processor -- start 2024-09-16T15:08:42.265 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.260+0000 7fad084a4640 1 -- start start 2024-09-16T15:08:42.266 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad00144790 0x7fad00148de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:42.266 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad0014b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:42.266 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0006fb60 con 0x7fad0006f170 2024-09-16T15:08:42.266 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad000b2ec0 con 0x7fad00144790 2024-09-16T15:08:42.266 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad000702f0 con 0x7fad000711a0 2024-09-16T15:08:42.266 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad05a3f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad00144790 0x7fad00148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:42.266 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad05a3f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fad00144790 0x7fad00148de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49842/0 (socket says 172.21.15.133:49842) 2024-09-16T15:08:42.266 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad05a3f640 1 -- 172.21.15.133:0/3824893505 learned_addr learned my addr 172.21.15.133:0/3824893505 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:42.267 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad06a41640 1 --2- 172.21.15.133:0/3824893505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad0014b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:42.267 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad05a3f640 1 -- 172.21.15.133:0/3824893505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 msgr2=0x7fad0014b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:42.267 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad05a3f640 1 --2- 172.21.15.133:0/3824893505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad0014b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:42.267 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad05a3f640 1 -- 172.21.15.133:0/3824893505 >> v1:172.21.15.191:6789/0 conn(0x7fad0006f170 legacy=0x7fad00144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:42.267 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad05a3f640 1 -- 172.21.15.133:0/3824893505 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000b1bd0 con 0x7fad00144790 2024-09-16T15:08:42.267 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad06a41640 1 --2- 172.21.15.133:0/3824893505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad0014b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:42.268 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad05a3f640 1 --2- 172.21.15.133:0/3824893505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad00144790 0x7fad00148de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7facf000bb10 tx=0x7facf001ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6903de2955f936d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:42.268 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad0523e640 1 -- 172.21.15.133:0/3824893505 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00a7980 con 0x7fad00144790 2024-09-16T15:08:42.268 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad0523e640 1 -- 172.21.15.133:0/3824893505 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf00a7ae0 con 0x7fad00144790 2024-09-16T15:08:42.269 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad0523e640 1 -- 172.21.15.133:0/3824893505 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00a7e70 con 0x7fad00144790 2024-09-16T15:08:42.269 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 -- 172.21.15.133:0/3824893505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad00144790 msgr2=0x7fad00148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:42.269 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 --2- 172.21.15.133:0/3824893505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad00144790 0x7fad00148de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7facf000bb10 tx=0x7facf001ea50 comp rx=0 tx=0).stop 2024-09-16T15:08:42.270 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 -- 172.21.15.133:0/3824893505 shutdown_connections 2024-09-16T15:08:42.270 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 --2- 172.21.15.133:0/3824893505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad0014b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:42.270 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 --2- 172.21.15.133:0/3824893505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad00144790 0x7fad00148de0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:42.270 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 -- 172.21.15.133:0/3824893505 >> 172.21.15.133:0/3824893505 conn(0x7fad000b1e60 msgr2=0x7fad000b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:42.270 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 -- 172.21.15.133:0/3824893505 shutdown_connections 2024-09-16T15:08:42.270 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.264+0000 7fad084a4640 1 -- 172.21.15.133:0/3824893505 wait complete. 2024-09-16T15:08:42.272 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 Processor -- start 2024-09-16T15:08:42.272 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 -- start start 2024-09-16T15:08:42.272 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad0006f170 0x7fad00140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:42.272 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad000a8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:42.272 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad00144790 0x7fad000a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:42.272 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000aa620 con 0x7fad0006f170 2024-09-16T15:08:42.272 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000a99d0 con 0x7fad00144790 2024-09-16T15:08:42.272 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad001df480 con 0x7fad000711a0 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad05a3f640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad000a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad06240640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad0006f170 0x7fad00140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad05a3f640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad000a8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:42342/0 (socket says 172.21.15.133:42342) 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad05a3f640 1 -- 172.21.15.133:0/2448026280 learned_addr learned my addr 172.21.15.133:0/2448026280 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad06a41640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad00144790 0x7fad000a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad05a3f640 1 -- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad00144790 msgr2=0x7fad000a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad05a3f640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad00144790 0x7fad000a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad05a3f640 1 -- 172.21.15.133:0/2448026280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad0006f170 msgr2=0x7fad00140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad05a3f640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad0006f170 0x7fad00140b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad05a3f640 1 -- 172.21.15.133:0/2448026280 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8009a70 con 0x7fad000711a0 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad06a41640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad00144790 0x7fad000a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad05a3f640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad000a8840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7facf001f2f0 tx=0x7facf001b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:42.273 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00af990 con 0x7fad000711a0 2024-09-16T15:08:42.274 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf00afb10 con 0x7fad000711a0 2024-09-16T15:08:42.274 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf000b770 con 0x7fad000711a0 2024-09-16T15:08:42.274 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00ba410 con 0x7fad000711a0 2024-09-16T15:08:42.274 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0014a500 con 0x7fad000711a0 2024-09-16T15:08:42.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.268+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad000b13b0 con 0x7fad000711a0 2024-09-16T15:08:42.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.272+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 10) v1 ==== 128936+0+0 (secure 0 0 0) 0x7facf00c3450 con 0x7fad000711a0 2024-09-16T15:08:42.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.272+0000 7facf6ffd640 20 mgrc handle_mgr_map mgrmap(e 10) v1 2024-09-16T15:08:42.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.272+0000 7facf6ffd640 4 mgrc handle_mgr_map Got map version 10 2024-09-16T15:08:42.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.272+0000 7facf6ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:42.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.272+0000 7facf6ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:42.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.272+0000 7facf6ffd640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7face4099de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:42.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.272+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5108+0+0 (secure 0 0 0) 0x7facf015a920 con 0x7fad000711a0 2024-09-16T15:08:42.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.272+0000 7fad06240640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7face4099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:42.276 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.272+0000 7fad06240640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7face4099de0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7facfc007fd0 tx=0x7facfc007e40 comp rx=0 tx=0).ready entity=mgr.4632 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:42.277 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.276+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7facf0127de0 con 0x7fad000711a0 2024-09-16T15:08:42.476 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.472+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "mgr module disable", "module": "localpool"} v 0) v1 -- 0x7fad000b6230 con 0x7fad000711a0 2024-09-16T15:08:42.882 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.880+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mgrmap(e 11) v1 ==== 128979+0+0 (secure 0 0 0) 0x7facf01281b0 con 0x7fad000711a0 2024-09-16T15:08:42.882 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.880+0000 7facf6ffd640 20 mgrc handle_mgr_map mgrmap(e 11) v1 2024-09-16T15:08:42.882 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.880+0000 7facf6ffd640 4 mgrc handle_mgr_map Got map version 11 2024-09-16T15:08:42.882 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.880+0000 7facf6ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:42.886 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.884+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 <== mon.2 v2:172.21.15.133:3301/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "localpool"}]=0 v11) v1 ==== 89+0+0 (secure 0 0 0) 0x7facf013f430 con 0x7fad000711a0 2024-09-16T15:08:42.914 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.912+0000 7fad06240640 1 -- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 msgr2=0x7face4099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-16T15:08:42.914 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.912+0000 7fad06240640 1 -- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 msgr2=0x7face4099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T15:08:42.915 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.912+0000 7fad06240640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7face4099de0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7facfc007fd0 tx=0x7facfc007e40 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T15:08:42.915 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.912+0000 7fad06240640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7face4099de0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7facfc007fd0 tx=0x7facfc007e40 comp rx=0 tx=0).stop 2024-09-16T15:08:42.915 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.912+0000 7facf6ffd640 4 mgrc ms_handle_reset ms_handle_reset con 0x7face4097920 2024-09-16T15:08:42.915 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.912+0000 7facf6ffd640 4 mgrc reconnect Terminating session with v2:172.21.15.191:6824/164170957 2024-09-16T15:08:42.915 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.912+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 msgr2=0x7face4099de0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T15:08:42.915 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.912+0000 7facf6ffd640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7face4099de0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:42.915 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:42.912+0000 7facf6ffd640 4 mgrc reconnect waiting to retry connect until 717.076831s 2024-09-16T15:08:43.022 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:43.020+0000 7f0c76800180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T15:08:43.032 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:43.027+0000 7fcde9e98180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T15:08:43.033 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:43.027+0000 7f596badf180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T15:08:43.046 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.044+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "mgr module disable", "module": "localpool"} v 0) v1 -- 0x7facdc005070 con 0x7fad000711a0 2024-09-16T15:08:43.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.272+0000 7facf57fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:43.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.272+0000 7facf57fa640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7fad00142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:43.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.272+0000 7fad06240640 1 -- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 msgr2=0x7fad00142600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:43.275 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.272+0000 7fad06240640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7fad00142600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:08:43.401 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:43.396+0000 7f0c76800180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T15:08:43.413 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:43.411+0000 7fcde9e98180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T15:08:43.414 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:43.411+0000 7f596badf180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T15:08:43.478 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.476+0000 7fad06240640 1 -- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 msgr2=0x7fad00142600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:43.478 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.476+0000 7fad06240640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7fad00142600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:08:43.531 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:43.528+0000 7f0c76800180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T15:08:43.546 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:43.543+0000 7f596badf180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T15:08:43.547 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:43.543+0000 7fcde9e98180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T15:08:43.882 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.880+0000 7fad06240640 1 -- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 msgr2=0x7fad00142600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:43.882 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.880+0000 7fad06240640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7fad00142600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:08:43.901 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.896+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 <== mon.2 v2:172.21.15.133:3301/0 9 ==== mgrmap(e 12) v1 ==== 128979+0+0 (secure 0 0 0) 0x7facf015c9d0 con 0x7fad000711a0 2024-09-16T15:08:43.901 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.896+0000 7facf6ffd640 20 mgrc handle_mgr_map mgrmap(e 12) v1 2024-09-16T15:08:43.901 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.896+0000 7facf6ffd640 4 mgrc handle_mgr_map Got map version 12 2024-09-16T15:08:43.901 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.896+0000 7facf6ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:43.902 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7facf6ffd640 1 -- 172.21.15.133:0/2448026280 <== mon.2 v2:172.21.15.133:3301/0 10 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "localpool"}]=0 module 'localpool' is already disabled v12) v1 ==== 127+0+0 (secure 0 0 0) 0x7facf0128570 con 0x7fad000711a0 2024-09-16T15:08:43.902 INFO:tasks.workunit.client.0.smithi133.stderr:module 'localpool' is already disabled 2024-09-16T15:08:43.905 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 10 mgrc shutdown 2024-09-16T15:08:43.905 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 msgr2=0x7fad00142600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:08:43.905 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7fad00142600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:43.905 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 msgr2=0x7fad000a8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:43.905 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad000a8840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7facf001f2f0 tx=0x7facf001b850 comp rx=0 tx=0).stop 2024-09-16T15:08:43.905 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 shutdown_connections 2024-09-16T15:08:43.905 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7face4097920 0x7fad00142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:43.906 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad00144790 0x7fad000a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:43.906 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad000711a0 0x7fad000a8840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:43.906 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 --2- 172.21.15.133:0/2448026280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad0006f170 0x7fad00140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:43.906 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 >> 172.21.15.133:0/2448026280 conn(0x7fad000b1e60 msgr2=0x7fad0013cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:43.906 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 shutdown_connections 2024-09-16T15:08:43.906 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:43.900+0000 7fad084a4640 1 -- 172.21.15.133:0/2448026280 wait complete. 2024-09-16T15:08:43.920 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph osd pool ls 2024-09-16T15:08:43.920 INFO:tasks.workunit.client.0.smithi133.stderr:+ grep ^by-host- 2024-09-16T15:08:43.957 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:43.952+0000 7f0c76800180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:44.002 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:43.998+0000 7fcde9e98180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:44.005 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.002+0000 7f596badf180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:44.009 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:44.008+0000 7f0c76800180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T15:08:44.029 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd6d7f5640 1 Processor -- start 2024-09-16T15:08:44.030 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd6d7f5640 1 -- start start 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd6d7f5640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcd680abc90 0x7fcd68071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd6d7f5640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd6806a5d0 0x7fcd6814e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd6d7f5640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd680a9110 con 0x7fcd680b5ae0 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd6d7f5640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd680a8b10 con 0x7fcd680abc90 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd6d7f5640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd680a8c80 con 0x7fcd6806a5d0 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd66ffd640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcd680abc90 0x7fcd68071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd5ffff640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcd680b5ae0 0x7fcd68069ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38384/0 (socket says 172.21.15.133:38384) 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd5ffff640 1 -- 172.21.15.133:0/842609421 learned_addr learned my addr 172.21.15.133:0/842609421 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd677fe640 1 --2- 172.21.15.133:0/842609421 >> v2:172.21.15.191:3300/0 conn(0x7fcd6806a5d0 0x7fcd6814e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd66ffd640 1 -- 172.21.15.133:0/842609421 >> v2:172.21.15.191:3300/0 conn(0x7fcd6806a5d0 msgr2=0x7fcd6814e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd66ffd640 1 --2- 172.21.15.133:0/842609421 >> v2:172.21.15.191:3300/0 conn(0x7fcd6806a5d0 0x7fcd6814e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd66ffd640 1 -- 172.21.15.133:0/842609421 >> v1:172.21.15.133:6789/0 conn(0x7fcd680b5ae0 legacy=0x7fcd68069ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:44.032 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd66ffd640 1 -- 172.21.15.133:0/842609421 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680aa660 con 0x7fcd680abc90 2024-09-16T15:08:44.033 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd66ffd640 1 --2- 172.21.15.133:0/842609421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd680abc90 0x7fcd68071610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcd680a6fd0 tx=0x7fcd5801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6798998208ab9816 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:44.033 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd667fc640 1 -- 172.21.15.133:0/842609421 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd580ab600 con 0x7fcd680abc90 2024-09-16T15:08:44.033 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd667fc640 1 -- 172.21.15.133:0/842609421 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd580a7050 con 0x7fcd680abc90 2024-09-16T15:08:44.033 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.028+0000 7fcd667fc640 1 -- 172.21.15.133:0/842609421 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd580ab980 con 0x7fcd680abc90 2024-09-16T15:08:44.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/842609421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd680abc90 msgr2=0x7fcd68071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:44.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 --2- 172.21.15.133:0/842609421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd680abc90 0x7fcd68071610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcd680a6fd0 tx=0x7fcd5801c3d0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/842609421 shutdown_connections 2024-09-16T15:08:44.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 --2- 172.21.15.133:0/842609421 >> v2:172.21.15.191:3300/0 conn(0x7fcd6806a5d0 0x7fcd6814e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 --2- 172.21.15.133:0/842609421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd680abc90 0x7fcd68071610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/842609421 >> 172.21.15.133:0/842609421 conn(0x7fcd68073150 msgr2=0x7fcd68073560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:44.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/842609421 shutdown_connections 2024-09-16T15:08:44.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/842609421 wait complete. 2024-09-16T15:08:44.036 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 Processor -- start 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- start start 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd6806a5d0 0x7fcd68147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd680abc90 0x7fcd68148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd680b5ae0 0x7fcd68148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd68140ee0 con 0x7fcd6806a5d0 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd68140960 con 0x7fcd680b5ae0 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd681df310 con 0x7fcd680abc90 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd66ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd6806a5d0 0x7fcd68147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd66ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd6806a5d0 0x7fcd68147610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49888/0 (socket says 172.21.15.133:49888) 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd66ffd640 1 -- 172.21.15.133:0/512878146 learned_addr learned my addr 172.21.15.133:0/512878146 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd5ffff640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd680abc90 0x7fcd68148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd677fe640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd680b5ae0 0x7fcd68148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:44.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd66ffd640 1 -- 172.21.15.133:0/512878146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd680abc90 msgr2=0x7fcd68148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:44.038 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd66ffd640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd680abc90 0x7fcd68148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.038 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd66ffd640 1 -- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd680b5ae0 msgr2=0x7fcd68148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:44.038 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd66ffd640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd680b5ae0 0x7fcd68148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.038 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd66ffd640 1 -- 172.21.15.133:0/512878146 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd58009420 con 0x7fcd6806a5d0 2024-09-16T15:08:44.038 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd66ffd640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd6806a5d0 0x7fcd68147610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd580ae040 tx=0x7fcd58019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:44.038 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd5f7fe640 1 -- 172.21.15.133:0/512878146 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd580aa8b0 con 0x7fcd6806a5d0 2024-09-16T15:08:44.038 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd5f7fe640 1 -- 172.21.15.133:0/512878146 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd580aaa30 con 0x7fcd6806a5d0 2024-09-16T15:08:44.038 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd5f7fe640 1 -- 172.21.15.133:0/512878146 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd580aade0 con 0x7fcd6806a5d0 2024-09-16T15:08:44.038 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6813b280 con 0x7fcd6806a5d0 2024-09-16T15:08:44.038 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.032+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd68147b50 con 0x7fcd6806a5d0 2024-09-16T15:08:44.039 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.036+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6813b410 con 0x7fcd6806a5d0 2024-09-16T15:08:44.039 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.036+0000 7fcd5f7fe640 1 -- 172.21.15.133:0/512878146 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) v1 ==== 128979+0+0 (secure 0 0 0) 0x7fcd580b2040 con 0x7fcd6806a5d0 2024-09-16T15:08:44.039 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.036+0000 7fcd5f7fe640 20 mgrc handle_mgr_map mgrmap(e 12) v1 2024-09-16T15:08:44.040 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.036+0000 7fcd5f7fe640 4 mgrc handle_mgr_map Got map version 12 2024-09-16T15:08:44.040 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.036+0000 7fcd5f7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:44.040 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.036+0000 7fcd5f7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:44.040 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.036+0000 7fcd5f7fe640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fcd48097990 0x7fcd48099e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.040 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.036+0000 7fcd5f7fe640 1 -- 172.21.15.133:0/512878146 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5537+0+0 (secure 0 0 0) 0x7fcd5815ac80 con 0x7fcd6806a5d0 2024-09-16T15:08:44.040 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.036+0000 7fcd5ffff640 1 -- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fcd48097990 msgr2=0x7fcd48099e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:44.040 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.036+0000 7fcd5ffff640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fcd48097990 0x7fcd48099e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:08:44.042 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.040+0000 7fcd5f7fe640 1 -- 172.21.15.133:0/512878146 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcd58127d60 con 0x7fcd6806a5d0 2024-09-16T15:08:44.054 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:44.050+0000 7fcde9e98180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T15:08:44.058 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.054+0000 7f596badf180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T15:08:44.061 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:44.060+0000 7f0c76800180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T15:08:44.104 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:44.098+0000 7fcde9e98180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T15:08:44.107 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.102+0000 7f596badf180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T15:08:44.136 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:44.132+0000 7f0c76800180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T15:08:44.179 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:44.174+0000 7fcde9e98180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T15:08:44.183 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.178+0000 7f596badf180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T15:08:44.186 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:44.184+0000 7f0c76800180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T15:08:44.204 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.200+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7fcd680a92b0 con 0x7fcd6806a5d0 2024-09-16T15:08:44.205 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.200+0000 7fcd5f7fe640 1 -- 172.21.15.133:0/512878146 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v30) v1 ==== 59+0+73 (secure 0 0 0) 0x7fcd5812b380 con 0x7fcd6806a5d0 2024-09-16T15:08:44.229 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:44.226+0000 7fcde9e98180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T15:08:44.233 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.230+0000 7f596badf180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T15:08:44.242 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.240+0000 7fcd5ffff640 1 -- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fcd48097990 msgr2=0x7fcd48099e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:44.242 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.240+0000 7fcd5ffff640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fcd48097990 0x7fcd48099e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:08:44.260 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:44.256+0000 7f0c76800180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T15:08:44.299 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.296+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7fcd680715e0 con 0x7fcd6806a5d0 2024-09-16T15:08:44.299 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.296+0000 7fcd5f7fe640 1 -- 172.21.15.133:0/512878146 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v30) v1 ==== 59+0+73 (secure 0 0 0) 0x7fcd580b0070 con 0x7fcd6806a5d0 2024-09-16T15:08:44.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.296+0000 7fcd6d7f5640 10 mgrc shutdown 2024-09-16T15:08:44.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fcd48097990 msgr2=0x7fcd48099e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:08:44.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fcd48097990 0x7fcd48099e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd6806a5d0 msgr2=0x7fcd68147610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:44.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd6806a5d0 0x7fcd68147610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd580ae040 tx=0x7fcd58019170 comp rx=0 tx=0).stop 2024-09-16T15:08:44.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 shutdown_connections 2024-09-16T15:08:44.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fcd48097990 0x7fcd48099e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd680b5ae0 0x7fcd68148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.302 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd680abc90 0x7fcd68148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.303 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 --2- 172.21.15.133:0/512878146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd6806a5d0 0x7fcd68147610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.303 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 >> 172.21.15.133:0/512878146 conn(0x7fcd68073150 msgr2=0x7fcd680b6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:44.303 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 shutdown_connections 2024-09-16T15:08:44.303 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.300+0000 7fcd6d7f5640 1 -- 172.21.15.133:0/512878146 wait complete. 2024-09-16T15:08:44.303 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:44.298+0000 7fcde9e98180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T15:08:44.307 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.302+0000 7f596badf180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T15:08:44.310 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:44.308+0000 7f0c76800180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T15:08:44.316 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph osd pool rm by-host-smithi133 by-host-smithi133 --yes-i-really-really-mean-it 2024-09-16T15:08:44.355 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:44.350+0000 7fcde9e98180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T15:08:44.359 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.354+0000 7f596badf180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T15:08:44.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20e5727640 1 Processor -- start 2024-09-16T15:08:44.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20e5727640 1 -- start start 2024-09-16T15:08:44.437 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20e5727640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 0x7f20e006aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.437 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20e5727640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f20e00aa000 0x7f20e00a8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.437 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20e5727640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00a8310 con 0x7f20e01413e0 2024-09-16T15:08:44.437 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20e5727640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e0141e60 con 0x7f20e00aa000 2024-09-16T15:08:44.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20e5727640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e0142120 con 0x7f20e0141000 2024-09-16T15:08:44.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20deffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 0x7f20e006aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:44.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20df7fe640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f20e01413e0 0x7f20e0144be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38396/0 (socket says 172.21.15.133:38396) 2024-09-16T15:08:44.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20df7fe640 1 -- 172.21.15.133:0/1337323300 learned_addr learned my addr 172.21.15.133:0/1337323300 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:08:44.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20de7fc640 1 --2- 172.21.15.133:0/1337323300 >> v2:172.21.15.191:3300/0 conn(0x7f20e00aa000 0x7f20e00a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:44.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20deffd640 1 -- 172.21.15.133:0/1337323300 >> v2:172.21.15.191:3300/0 conn(0x7f20e00aa000 msgr2=0x7f20e00a8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:44.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20deffd640 1 --2- 172.21.15.133:0/1337323300 >> v2:172.21.15.191:3300/0 conn(0x7f20e00aa000 0x7f20e00a8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20deffd640 1 -- 172.21.15.133:0/1337323300 >> v1:172.21.15.133:6789/0 conn(0x7f20e01413e0 legacy=0x7f20e0144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:44.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20ddffb640 1 -- 172.21.15.133:0/1337323300 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228156122 0 0) 0x7f20e00a8310 con 0x7f20e01413e0 2024-09-16T15:08:44.438 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.432+0000 7f20deffd640 1 -- 172.21.15.133:0/1337323300 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e00a9890 con 0x7f20e0141000 2024-09-16T15:08:44.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20deffd640 1 --2- 172.21.15.133:0/1337323300 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 0x7f20e006aed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009d90 tx=0x7f20d801f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e855bd8e88cc7725 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:44.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20ddffb640 1 -- 172.21.15.133:0/1337323300 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d80af600 con 0x7f20e0141000 2024-09-16T15:08:44.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20ddffb640 1 -- 172.21.15.133:0/1337323300 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d80ab050 con 0x7f20e0141000 2024-09-16T15:08:44.439 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20ddffb640 1 -- 172.21.15.133:0/1337323300 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d80af980 con 0x7f20e0141000 2024-09-16T15:08:44.440 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 -- 172.21.15.133:0/1337323300 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 msgr2=0x7f20e006aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:44.440 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 --2- 172.21.15.133:0/1337323300 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 0x7f20e006aed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009d90 tx=0x7f20d801f130 comp rx=0 tx=0).stop 2024-09-16T15:08:44.440 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 -- 172.21.15.133:0/1337323300 shutdown_connections 2024-09-16T15:08:44.440 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 --2- 172.21.15.133:0/1337323300 >> v2:172.21.15.191:3300/0 conn(0x7f20e00aa000 0x7f20e00a8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.440 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 --2- 172.21.15.133:0/1337323300 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 0x7f20e006aed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.440 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 -- 172.21.15.133:0/1337323300 >> 172.21.15.133:0/1337323300 conn(0x7f20e00b3ef0 msgr2=0x7f20e00b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:44.441 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 -- 172.21.15.133:0/1337323300 shutdown_connections 2024-09-16T15:08:44.441 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 -- 172.21.15.133:0/1337323300 wait complete. 2024-09-16T15:08:44.442 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 Processor -- start 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 -- start start 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e00aa000 0x7f20e01da050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 0x7f20e00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20e01413e0 0x7f20e00b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00b0e90 con 0x7f20e00aa000 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00ad1d0 con 0x7f20e01413e0 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.436+0000 7f20e5727640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e00ad430 con 0x7f20e0141000 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20deffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e00aa000 0x7f20e01da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20deffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e00aa000 0x7f20e01da050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49904/0 (socket says 172.21.15.133:49904) 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20deffd640 1 -- 172.21.15.133:0/900594215 learned_addr learned my addr 172.21.15.133:0/900594215 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20de7fc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 0x7f20e00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20df7fe640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20e01413e0 0x7f20e00b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20deffd640 1 -- 172.21.15.133:0/900594215 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 msgr2=0x7f20e00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:44.443 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20deffd640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 0x7f20e00b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20deffd640 1 -- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20e01413e0 msgr2=0x7f20e00b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:44.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20deffd640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20e01413e0 0x7f20e00b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20deffd640 1 -- 172.21.15.133:0/900594215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d8009a70 con 0x7f20e00aa000 2024-09-16T15:08:44.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20de7fc640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 0x7f20e00b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:44.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20deffd640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e00aa000 0x7f20e01da050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009d90 tx=0x7f20d801bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:44.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20c77fe640 1 -- 172.21.15.133:0/900594215 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8003ba0 con 0x7f20e00aa000 2024-09-16T15:08:44.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20c77fe640 1 -- 172.21.15.133:0/900594215 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8003d00 con 0x7f20e00aa000 2024-09-16T15:08:44.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e01da970 con 0x7f20e00aa000 2024-09-16T15:08:44.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20c77fe640 1 -- 172.21.15.133:0/900594215 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8004230 con 0x7f20e00aa000 2024-09-16T15:08:44.444 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e01d65d0 con 0x7f20e00aa000 2024-09-16T15:08:44.445 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e013c5a0 con 0x7f20e00aa000 2024-09-16T15:08:44.445 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20c77fe640 1 -- 172.21.15.133:0/900594215 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) v1 ==== 128979+0+0 (secure 0 0 0) 0x7f20d8004390 con 0x7f20e00aa000 2024-09-16T15:08:44.445 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20c77fe640 20 mgrc handle_mgr_map mgrmap(e 12) v1 2024-09-16T15:08:44.445 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20c77fe640 4 mgrc handle_mgr_map Got map version 12 2024-09-16T15:08:44.446 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20c77fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:44.446 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20c77fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:44.446 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20c77fe640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7f20cc097960 0x7f20cc099e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:44.446 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20de7fc640 1 -- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7f20cc097960 msgr2=0x7f20cc099e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:44.446 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20c77fe640 1 -- 172.21.15.133:0/900594215 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5537+0+0 (secure 0 0 0) 0x7f20d815efc0 con 0x7f20e00aa000 2024-09-16T15:08:44.446 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.440+0000 7f20de7fc640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7f20cc097960 0x7f20cc099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:08:44.448 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.444+0000 7f20c77fe640 1 -- 172.21.15.133:0/900594215 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f20d812fbc0 con 0x7f20e00aa000 2024-09-16T15:08:44.455 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:44.452+0000 7f0c76800180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T15:08:44.497 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:44.494+0000 7fcde9e98180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T15:08:44.500 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.494+0000 7f596badf180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T15:08:44.508 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:44.504+0000 7f0c76800180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T15:08:44.549 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:44.546+0000 7fcde9e98180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T15:08:44.552 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.546+0000 7f596badf180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T15:08:44.609 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.608+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "by-host-smithi133", "pool2": "by-host-smithi133", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f20e00aa540 con 0x7f20e00aa000 2024-09-16T15:08:44.646 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.644+0000 7f20de7fc640 1 -- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7f20cc097960 msgr2=0x7f20cc099e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:44.646 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.644+0000 7f20de7fc640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7f20cc097960 0x7f20cc099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:08:44.725 INFO:tasks.ceph.mgr.x.smithi133.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T15:08:44.725 INFO:tasks.ceph.mgr.x.smithi133.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T15:08:44.725 INFO:tasks.ceph.mgr.x.smithi133.stderr: from numpy import show_config as show_numpy_config 2024-09-16T15:08:44.732 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:44.728+0000 7f0c76800180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T15:08:44.768 INFO:tasks.ceph.mgr.y.smithi191.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T15:08:44.768 INFO:tasks.ceph.mgr.y.smithi191.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T15:08:44.768 INFO:tasks.ceph.mgr.y.smithi191.stderr: from numpy import show_config as show_numpy_config 2024-09-16T15:08:44.773 INFO:tasks.ceph.mgr.z.smithi191.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-16T15:08:44.773 INFO:tasks.ceph.mgr.z.smithi191.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-16T15:08:44.773 INFO:tasks.ceph.mgr.z.smithi191.stderr: from numpy import show_config as show_numpy_config 2024-09-16T15:08:44.776 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:44.770+0000 7fcde9e98180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T15:08:44.781 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.778+0000 7f596badf180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T15:08:44.795 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.792+0000 7f20c77fe640 1 -- 172.21.15.133:0/900594215 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "by-host-smithi133", "pool2": "by-host-smithi133", "yes_i_really_really_mean_it": true}]=0 pool 'by-host-smithi133' removed v31) v1 ==== 187+0+0 (secure 0 0 0) 0x7f20d8009450 con 0x7f20e00aa000 2024-09-16T15:08:44.893 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:44.888+0000 7f0c76800180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:44.920 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.916+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "by-host-smithi133", "pool2": "by-host-smithi133", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f20e00b4d20 con 0x7f20e00aa000 2024-09-16T15:08:44.920 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.916+0000 7f20c77fe640 1 -- 172.21.15.133:0/900594215 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "by-host-smithi133", "pool2": "by-host-smithi133", "yes_i_really_really_mean_it": true}]=0 pool 'by-host-smithi133' does not exist v31) v1 ==== 194+0+0 (secure 0 0 0) 0x7f20d80b4020 con 0x7f20e00aa000 2024-09-16T15:08:44.920 INFO:tasks.workunit.client.0.smithi133.stderr:pool 'by-host-smithi133' does not exist 2024-09-16T15:08:44.923 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 10 mgrc shutdown 2024-09-16T15:08:44.923 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7f20cc097960 msgr2=0x7f20cc099e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7f20cc097960 0x7f20cc099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e00aa000 msgr2=0x7f20e01da050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e00aa000 0x7f20e01da050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009d90 tx=0x7f20d801bed0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 shutdown_connections 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7f20cc097960 0x7f20cc099e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20e01413e0 0x7f20e00b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20e0141000 0x7f20e00b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 --2- 172.21.15.133:0/900594215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20e00aa000 0x7f20e01da050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 >> 172.21.15.133:0/900594215 conn(0x7f20e00b3ef0 msgr2=0x7f20e0069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 shutdown_connections 2024-09-16T15:08:44.924 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:44.920+0000 7f20e5727640 1 -- 172.21.15.133:0/900594215 wait complete. 2024-09-16T15:08:44.937 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph osd pool rm by-host-smithi191 by-host-smithi191 --yes-i-really-really-mean-it 2024-09-16T15:08:44.957 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:44.954+0000 7fcde9e98180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:44.959 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:44.954+0000 7f596badf180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:45.006 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:45.004+0000 7f0c76800180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:45.051 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feffb765640 1 Processor -- start 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feffb765640 1 -- start start 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feffb765640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feff406dd70 0x7feff40a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feffb765640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7feff406e470 0x7feff41474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feffb765640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff40b7af0 con 0x7feff40a8b50 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feffb765640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff40aa290 con 0x7feff406dd70 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feffb765640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff40b7db0 con 0x7feff406e470 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feff8d00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feff406dd70 0x7feff40a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feff9501640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7feff40a8b50 0x7feff4146dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38398/0 (socket says 172.21.15.133:38398) 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feff8d00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feff406dd70 0x7feff40a9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49908/0 (socket says 172.21.15.133:49908) 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feff9501640 1 -- 172.21.15.133:0/1017380454 learned_addr learned my addr 172.21.15.133:0/1017380454 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feff9d02640 1 --2- 172.21.15.133:0/1017380454 >> v2:172.21.15.191:3300/0 conn(0x7feff406e470 0x7feff41474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feff8d00640 1 -- 172.21.15.133:0/1017380454 >> v2:172.21.15.191:3300/0 conn(0x7feff406e470 msgr2=0x7feff41474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feff8d00640 1 --2- 172.21.15.133:0/1017380454 >> v2:172.21.15.191:3300/0 conn(0x7feff406e470 0x7feff41474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.052 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feff8d00640 1 -- 172.21.15.133:0/1017380454 >> v1:172.21.15.133:6789/0 conn(0x7feff40a8b50 legacy=0x7feff4146dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:45.053 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7feff8d00640 1 -- 172.21.15.133:0/1017380454 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff406e8e0 con 0x7feff406dd70 2024-09-16T15:08:45.053 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.048+0000 7fefe7fff640 1 -- 172.21.15.133:0/1017380454 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3829804995 0 0) 0x7feff40b7af0 con 0x7feff40a8b50 2024-09-16T15:08:45.054 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff8d00640 1 --2- 172.21.15.133:0/1017380454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 0x7feff40a9d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefe80097c0 tx=0x7fefe801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc8e3e548b16594 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:45.054 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7fefe7fff640 1 -- 172.21.15.133:0/1017380454 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe80ab600 con 0x7feff406dd70 2024-09-16T15:08:45.054 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7fefe7fff640 1 -- 172.21.15.133:0/1017380454 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe80a7050 con 0x7feff406dd70 2024-09-16T15:08:45.054 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7fefe7fff640 1 -- 172.21.15.133:0/1017380454 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe80ab980 con 0x7feff406dd70 2024-09-16T15:08:45.055 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- 172.21.15.133:0/1017380454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 msgr2=0x7feff40a9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:45.055 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 --2- 172.21.15.133:0/1017380454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 0x7feff40a9d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fefe80097c0 tx=0x7fefe801c3b0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.055 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- 172.21.15.133:0/1017380454 shutdown_connections 2024-09-16T15:08:45.055 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 --2- 172.21.15.133:0/1017380454 >> v2:172.21.15.191:3300/0 conn(0x7feff406e470 0x7feff41474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.055 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 --2- 172.21.15.133:0/1017380454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 0x7feff40a9d50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.055 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- 172.21.15.133:0/1017380454 >> 172.21.15.133:0/1017380454 conn(0x7feff406f4b0 msgr2=0x7feff4069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:45.055 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- 172.21.15.133:0/1017380454 shutdown_connections 2024-09-16T15:08:45.055 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- 172.21.15.133:0/1017380454 wait complete. 2024-09-16T15:08:45.056 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 Processor -- start 2024-09-16T15:08:45.056 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- start start 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 0x7feff413ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff406e470 0x7feff4143a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feff40a8b50 0x7feff40a6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff4143f50 con 0x7feff406dd70 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff4142350 con 0x7feff406e470 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff4142610 con 0x7feff40a8b50 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9501640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 0x7feff413ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9d02640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feff40a8b50 0x7feff40a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9501640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 0x7feff413ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49918/0 (socket says 172.21.15.133:49918) 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9501640 1 -- 172.21.15.133:0/2589794947 learned_addr learned my addr 172.21.15.133:0/2589794947 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff8d00640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff406e470 0x7feff4143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9501640 1 -- 172.21.15.133:0/2589794947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feff40a8b50 msgr2=0x7feff40a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:45.058 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9501640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feff40a8b50 0x7feff40a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.059 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9501640 1 -- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff406e470 msgr2=0x7feff4143a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:45.059 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9501640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff406e470 0x7feff4143a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.059 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9501640 1 -- 172.21.15.133:0/2589794947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe8009420 con 0x7feff406dd70 2024-09-16T15:08:45.059 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9d02640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feff40a8b50 0x7feff40a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:45.059 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feff9501640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 0x7feff413ee50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feff000c480 tx=0x7feff000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:45.059 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7fefe5ffb640 1 -- 172.21.15.133:0/2589794947 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff00074e0 con 0x7feff406dd70 2024-09-16T15:08:45.059 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7fefe5ffb640 1 -- 172.21.15.133:0/2589794947 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff0007660 con 0x7feff406dd70 2024-09-16T15:08:45.059 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff413f790 con 0x7feff406dd70 2024-09-16T15:08:45.059 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7fefe5ffb640 1 -- 172.21.15.133:0/2589794947 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0004a30 con 0x7feff406dd70 2024-09-16T15:08:45.059 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.052+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff413a130 con 0x7feff406dd70 2024-09-16T15:08:45.060 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.056+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff40a7c30 con 0x7feff406dd70 2024-09-16T15:08:45.060 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.056+0000 7fefe5ffb640 1 -- 172.21.15.133:0/2589794947 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) v1 ==== 128979+0+0 (secure 0 0 0) 0x7feff0004b90 con 0x7feff406dd70 2024-09-16T15:08:45.060 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.056+0000 7fefe5ffb640 20 mgrc handle_mgr_map mgrmap(e 12) v1 2024-09-16T15:08:45.060 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.056+0000 7fefe5ffb640 4 mgrc handle_mgr_map Got map version 12 2024-09-16T15:08:45.060 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.056+0000 7fefe5ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:45.060 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.056+0000 7fefe5ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:45.060 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.056+0000 7fefe5ffb640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fefd8097960 0x7fefd8099e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:45.060 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.056+0000 7fefe5ffb640 1 -- 172.21.15.133:0/2589794947 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5181+0+0 (secure 0 0 0) 0x7feff00bf1f0 con 0x7feff406dd70 2024-09-16T15:08:45.060 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.056+0000 7feff8d00640 1 -- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fefd8097960 msgr2=0x7fefd8099e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:45.061 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.056+0000 7feff8d00640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fefd8097960 0x7fefd8099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:08:45.064 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.060+0000 7fefe5ffb640 1 -- 172.21.15.133:0/2589794947 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feff0090090 con 0x7feff406dd70 2024-09-16T15:08:45.068 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:45.062+0000 7fcde9e98180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:45.070 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:45.066+0000 7f596badf180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T15:08:45.074 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:45.072+0000 7f0c76800180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T15:08:45.131 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:45.126+0000 7fcde9e98180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T15:08:45.132 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:45.126+0000 7f596badf180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T15:08:45.213 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:45.208+0000 7f0c76800180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:45.235 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.232+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "by-host-smithi191", "pool2": "by-host-smithi191", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feff4142e80 con 0x7feff406dd70 2024-09-16T15:08:45.262 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.260+0000 7feff8d00640 1 -- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fefd8097960 msgr2=0x7fefd8099e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:45.262 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.260+0000 7feff8d00640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fefd8097960 0x7fefd8099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:08:45.264 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:45.258+0000 7fcde9e98180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:45.265 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:45.262+0000 7f596badf180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T15:08:45.666 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.664+0000 7feff8d00640 1 -- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fefd8097960 msgr2=0x7fefd8099e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:45.666 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.664+0000 7feff8d00640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fefd8097960 0x7fefd8099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T15:08:45.682 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:45.680+0000 7f0c76800180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T15:08:45.750 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:45.746+0000 7fcde9e98180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T15:08:45.751 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:45.746+0000 7f596badf180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T15:08:45.754 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:45.752+0000 7f0c76800180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T15:08:45.795 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.792+0000 7fefe5ffb640 1 -- 172.21.15.133:0/2589794947 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "by-host-smithi191", "pool2": "by-host-smithi191", "yes_i_really_really_mean_it": true}]=0 pool 'by-host-smithi191' removed v32) v1 ==== 187+0+0 (secure 0 0 0) 0x7feff00ace60 con 0x7feff406dd70 2024-09-16T15:08:45.845 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:45.842+0000 7fcde9e98180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T15:08:45.845 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:45.842+0000 7f596badf180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T15:08:45.891 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.888+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "by-host-smithi191", "pool2": "by-host-smithi191", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feff40b6a20 con 0x7feff406dd70 2024-09-16T15:08:45.892 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.888+0000 7fefe5ffb640 1 -- 172.21.15.133:0/2589794947 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "by-host-smithi191", "pool2": "by-host-smithi191", "yes_i_really_really_mean_it": true}]=0 pool 'by-host-smithi191' does not exist v32) v1 ==== 194+0+0 (secure 0 0 0) 0x7feff0018070 con 0x7feff406dd70 2024-09-16T15:08:45.892 INFO:tasks.workunit.client.0.smithi133.stderr:pool 'by-host-smithi191' does not exist 2024-09-16T15:08:45.895 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 10 mgrc shutdown 2024-09-16T15:08:45.895 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fefd8097960 msgr2=0x7fefd8099e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:08:45.895 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fefd8097960 0x7fefd8099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.895 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 msgr2=0x7feff413ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:45.895 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 0x7feff413ee50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7feff000c480 tx=0x7feff000c940 comp rx=0 tx=0).stop 2024-09-16T15:08:45.895 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 shutdown_connections 2024-09-16T15:08:45.895 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fefd8097960 0x7fefd8099e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.895 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feff40a8b50 0x7feff40a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.895 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff406e470 0x7feff4143a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.896 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 --2- 172.21.15.133:0/2589794947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feff406dd70 0x7feff413ee50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:45.896 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 >> 172.21.15.133:0/2589794947 conn(0x7feff406f4b0 msgr2=0x7feff406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:45.896 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 shutdown_connections 2024-09-16T15:08:45.896 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:45.892+0000 7feffb765640 1 -- 172.21.15.133:0/2589794947 wait complete. 2024-09-16T15:08:45.902 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:45.900+0000 7f0c76800180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T15:08:45.909 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph config rm mgr mgr/localpool/subtree 2024-09-16T15:08:45.980 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:45.976+0000 7f0c76800180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T15:08:46.020 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:46.018+0000 7fcde9e98180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T15:08:46.025 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.020+0000 7fc733631640 1 Processor -- start 2024-09-16T15:08:46.025 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.020+0000 7fc733631640 1 -- start start 2024-09-16T15:08:46.026 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc733631640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc72c06b590 0x7fc72c071f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.027 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc733631640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72c073560 con 0x7fc72c06aa60 2024-09-16T15:08:46.027 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc733631640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72c06aef0 con 0x7fc72c073180 2024-09-16T15:08:46.027 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc733631640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc72c0b4be0 con 0x7fc72c06b590 2024-09-16T15:08:46.027 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc731bce640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc72c06b590 0x7fc72c071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:46.027 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc730bcc640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc72c06aa60 0x7fc72c148b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:38402/0 (socket says 172.21.15.133:38402) 2024-09-16T15:08:46.027 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc730bcc640 1 -- 172.21.15.133:0/300020547 learned_addr learned my addr 172.21.15.133:0/300020547 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:08:46.027 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc731bce640 1 -- 172.21.15.133:0/300020547 >> v1:172.21.15.191:6789/0 conn(0x7fc72c073180 legacy=0x7fc72c140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:46.027 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc731bce640 1 -- 172.21.15.133:0/300020547 >> v1:172.21.15.133:6789/0 conn(0x7fc72c06aa60 legacy=0x7fc72c148b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:46.027 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc723fff640 1 -- 172.21.15.133:0/300020547 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241442461 0 0) 0x7fc72c073560 con 0x7fc72c06aa60 2024-09-16T15:08:46.027 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.024+0000 7fc731bce640 1 -- 172.21.15.133:0/300020547 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c0b4550 con 0x7fc72c06b590 2024-09-16T15:08:46.027 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.022+0000 7f596badf180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T15:08:46.030 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc731bce640 1 --2- 172.21.15.133:0/300020547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06b590 0x7fc72c071f80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc72400bfa0 tx=0x7fc72401ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5841f9e045f17d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:46.030 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc723fff640 1 -- 172.21.15.133:0/300020547 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72400b2d0 con 0x7fc72c06b590 2024-09-16T15:08:46.030 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc723fff640 1 -- 172.21.15.133:0/300020547 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7240a7710 con 0x7fc72c06b590 2024-09-16T15:08:46.030 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc723fff640 1 -- 172.21.15.133:0/300020547 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7240a7aa0 con 0x7fc72c06b590 2024-09-16T15:08:46.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 -- 172.21.15.133:0/300020547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06b590 msgr2=0x7fc72c071f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:46.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 --2- 172.21.15.133:0/300020547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06b590 0x7fc72c071f80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc72400bfa0 tx=0x7fc72401ea30 comp rx=0 tx=0).stop 2024-09-16T15:08:46.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 -- 172.21.15.133:0/300020547 shutdown_connections 2024-09-16T15:08:46.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 --2- 172.21.15.133:0/300020547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06b590 0x7fc72c071f80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 -- 172.21.15.133:0/300020547 >> 172.21.15.133:0/300020547 conn(0x7fc72c0b5ca0 msgr2=0x7fc72c0b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:46.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 -- 172.21.15.133:0/300020547 shutdown_connections 2024-09-16T15:08:46.031 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 -- 172.21.15.133:0/300020547 wait complete. 2024-09-16T15:08:46.032 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 Processor -- start 2024-09-16T15:08:46.032 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 -- start start 2024-09-16T15:08:46.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06aa60 0x7fc72c0a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc72c06b590 0x7fc72c0af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc72c073180 0x7fc72c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c144190 con 0x7fc72c06aa60 2024-09-16T15:08:46.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c0b19e0 con 0x7fc72c073180 2024-09-16T15:08:46.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc733631640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72c13a450 con 0x7fc72c06b590 2024-09-16T15:08:46.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc7313cd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06aa60 0x7fc72c0a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:46.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc7313cd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06aa60 0x7fc72c0a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49928/0 (socket says 172.21.15.133:49928) 2024-09-16T15:08:46.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc7313cd640 1 -- 172.21.15.133:0/4137220449 learned_addr learned my addr 172.21.15.133:0/4137220449 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:46.034 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc730bcc640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc72c06b590 0x7fc72c0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc731bce640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc72c073180 0x7fc72c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc730bcc640 1 -- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc72c073180 msgr2=0x7fc72c0af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc730bcc640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc72c073180 0x7fc72c0af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc730bcc640 1 -- 172.21.15.133:0/4137220449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06aa60 msgr2=0x7fc72c0a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc730bcc640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06aa60 0x7fc72c0a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc730bcc640 1 -- 172.21.15.133:0/4137220449 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72400bc80 con 0x7fc72c06b590 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc7313cd640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06aa60 0x7fc72c0a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.028+0000 7fc730bcc640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc72c06b590 0x7fc72c0af250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc71c000c10 tx=0x7fc71c00c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc721ffb640 1 -- 172.21.15.133:0/4137220449 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0074e0 con 0x7fc72c06b590 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc721ffb640 1 -- 172.21.15.133:0/4137220449 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c007640 con 0x7fc72c06b590 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72c144660 con 0x7fc72c06b590 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc721ffb640 1 -- 172.21.15.133:0/4137220449 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c004a30 con 0x7fc72c06b590 2024-09-16T15:08:46.035 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72c0a51f0 con 0x7fc72c06b590 2024-09-16T15:08:46.036 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72c0b5ab0 con 0x7fc72c06b590 2024-09-16T15:08:46.036 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc721ffb640 1 -- 172.21.15.133:0/4137220449 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 12) v1 ==== 128979+0+0 (secure 0 0 0) 0x7fc71c004b90 con 0x7fc72c06b590 2024-09-16T15:08:46.036 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc721ffb640 20 mgrc handle_mgr_map mgrmap(e 12) v1 2024-09-16T15:08:46.036 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc721ffb640 4 mgrc handle_mgr_map Got map version 12 2024-09-16T15:08:46.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc721ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:46.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc721ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:46.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc721ffb640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fc710097960 0x7fc710099e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc7313cd640 1 -- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fc710097960 msgr2=0x7fc710099e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:46.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc721ffb640 1 -- 172.21.15.133:0/4137220449 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4825+0+0 (secure 0 0 0) 0x7fc71c0c03e0 con 0x7fc72c06b590 2024-09-16T15:08:46.037 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.032+0000 7fc7313cd640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fc710097960 0x7fc710099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:08:46.043 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.040+0000 7fc721ffb640 1 -- 172.21.15.133:0/4137220449 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc71c08d810 con 0x7fc72c06b590 2024-09-16T15:08:46.095 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:46.090+0000 7fcde9e98180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T15:08:46.101 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.098+0000 7f596badf180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T15:08:46.193 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.188+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/subtree"} v 0) v1 -- 0x7fc72c0a6350 con 0x7fc72c06b590 2024-09-16T15:08:46.199 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.196+0000 7fc721ffb640 1 -- 172.21.15.133:0/4137220449 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/subtree"}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc71c08f740 con 0x7fc72c06b590 2024-09-16T15:08:46.238 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.236+0000 7fc7313cd640 1 -- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fc710097960 msgr2=0x7fc710099e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:46.238 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.236+0000 7fc7313cd640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fc710097960 0x7fc710099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T15:08:46.268 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:46.264+0000 7f0c76800180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T15:08:46.293 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.288+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/subtree"} v 0) v1 -- 0x7fc72c06f0a0 con 0x7fc72c06b590 2024-09-16T15:08:46.294 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc721ffb640 1 -- 172.21.15.133:0/4137220449 <== mon.2 v2:172.21.15.133:3301/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/subtree"}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc71c090ec0 con 0x7fc72c06b590 2024-09-16T15:08:46.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 10 mgrc shutdown 2024-09-16T15:08:46.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fc710097960 msgr2=0x7fc710099e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:08:46.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fc710097960 0x7fc710099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc72c06b590 msgr2=0x7fc72c0af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:46.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc72c06b590 0x7fc72c0af250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc71c000c10 tx=0x7fc71c00c8e0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 shutdown_connections 2024-09-16T15:08:46.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fc710097960 0x7fc710099e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc72c073180 0x7fc72c0af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc72c06b590 0x7fc72c0af250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.297 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 --2- 172.21.15.133:0/4137220449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc72c06aa60 0x7fc72c0a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.298 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 >> 172.21.15.133:0/4137220449 conn(0x7fc72c0b5ca0 msgr2=0x7fc72c071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:46.298 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 shutdown_connections 2024-09-16T15:08:46.298 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.292+0000 7fc733631640 1 -- 172.21.15.133:0/4137220449 wait complete. 2024-09-16T15:08:46.311 INFO:tasks.workunit.client.0.smithi133.stderr:+ ceph config rm mgr mgr/localpool/failure_domain 2024-09-16T15:08:46.324 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-16T15:08:46.320+0000 7f0c76800180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T15:08:46.379 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:46.374+0000 7fcde9e98180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T15:08:46.387 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.382+0000 7f596badf180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T15:08:46.423 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccde0e1640 1 Processor -- start 2024-09-16T15:08:46.423 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccde0e1640 1 -- start start 2024-09-16T15:08:46.424 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccde0e1640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8140b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.424 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccde0e1640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fccd8141040 0x7fccd814bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccde0e1640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd8141850 con 0x7fccd8070210 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccde0e1640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd8141af0 con 0x7fccd8141040 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccde0e1640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8071d40 con 0x7fccd80b1820 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccd7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fccd8141040 0x7fccd814bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccd6ffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccd7fff640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fccd8141040 0x7fccd814bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49938/0 (socket says 172.21.15.133:49938) 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccd7fff640 1 -- 172.21.15.133:0/3940621024 learned_addr learned my addr 172.21.15.133:0/3940621024 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccd6ffd640 1 -- 172.21.15.133:0/3940621024 >> v2:172.21.15.133:3300/0 conn(0x7fccd8141040 msgr2=0x7fccd814bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccd6ffd640 1 --2- 172.21.15.133:0/3940621024 >> v2:172.21.15.133:3300/0 conn(0x7fccd8141040 0x7fccd814bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccd6ffd640 1 -- 172.21.15.133:0/3940621024 >> v1:172.21.15.191:6789/0 conn(0x7fccd8070210 legacy=0x7fccd8071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccd6ffd640 1 -- 172.21.15.133:0/3940621024 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd80b2e90 con 0x7fccd80b1820 2024-09-16T15:08:46.425 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.420+0000 7fccd7fff640 1 --2- 172.21.15.133:0/3940621024 >> v2:172.21.15.133:3300/0 conn(0x7fccd8141040 0x7fccd814bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:46.428 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccd6ffd640 1 --2- 172.21.15.133:0/3940621024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8140b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0097c0 tx=0x7fcccc01c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=137bbba872cbcdd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:46.428 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccd67fc640 1 -- 172.21.15.133:0/3940621024 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0ab600 con 0x7fccd80b1820 2024-09-16T15:08:46.428 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccd67fc640 1 -- 172.21.15.133:0/3940621024 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc0a7050 con 0x7fccd80b1820 2024-09-16T15:08:46.428 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccd67fc640 1 -- 172.21.15.133:0/3940621024 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0ab980 con 0x7fccd80b1820 2024-09-16T15:08:46.429 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccde0e1640 1 -- 172.21.15.133:0/3940621024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 msgr2=0x7fccd8140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:46.429 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccde0e1640 1 --2- 172.21.15.133:0/3940621024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8140b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0097c0 tx=0x7fcccc01c360 comp rx=0 tx=0).stop 2024-09-16T15:08:46.429 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccde0e1640 1 -- 172.21.15.133:0/3940621024 shutdown_connections 2024-09-16T15:08:46.429 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccde0e1640 1 --2- 172.21.15.133:0/3940621024 >> v2:172.21.15.133:3300/0 conn(0x7fccd8141040 0x7fccd814bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.429 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccde0e1640 1 --2- 172.21.15.133:0/3940621024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8140b00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.429 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccde0e1640 1 -- 172.21.15.133:0/3940621024 >> 172.21.15.133:0/3940621024 conn(0x7fccd80b1e60 msgr2=0x7fccd80b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:46.430 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccde0e1640 1 -- 172.21.15.133:0/3940621024 shutdown_connections 2024-09-16T15:08:46.430 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.424+0000 7fccde0e1640 1 -- 172.21.15.133:0/3940621024 wait complete. 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 Processor -- start 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 -- start start 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd8070210 0x7fccd806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8137f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd8141040 0x7fccd806e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8138460 con 0x7fccd8070210 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd813dc10 con 0x7fccd8141040 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8138610 con 0x7fccd80b1820 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd6ffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8137f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd8070210 0x7fccd806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd6ffd640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8137f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:42408/0 (socket says 172.21.15.133:42408) 2024-09-16T15:08:46.432 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd6ffd640 1 -- 172.21.15.133:0/2049579518 learned_addr learned my addr 172.21.15.133:0/2049579518 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd7fff640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd8141040 0x7fccd806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd6ffd640 1 -- 172.21.15.133:0/2049579518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd8141040 msgr2=0x7fccd806e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd6ffd640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd8141040 0x7fccd806e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd6ffd640 1 -- 172.21.15.133:0/2049579518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd8070210 msgr2=0x7fccd806f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd6ffd640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd8070210 0x7fccd806f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd6ffd640 1 -- 172.21.15.133:0/2049579518 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd0009b30 con 0x7fccd80b1820 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd77fe640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd8070210 0x7fccd806f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccd6ffd640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8137f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcccc019f90 tx=0x7fcccc019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccbffff640 1 -- 172.21.15.133:0/2049579518 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0abdd0 con 0x7fccd80b1820 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccbffff640 1 -- 172.21.15.133:0/2049579518 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc0b4430 con 0x7fccd80b1820 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 -- 172.21.15.133:0/2049579518 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc009420 con 0x7fccd80b1820 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccbffff640 1 -- 172.21.15.133:0/2049579518 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0b47e0 con 0x7fccd80b1820 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 -- 172.21.15.133:0/2049579518 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd806e960 con 0x7fccd80b1820 2024-09-16T15:08:46.433 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccde0e1640 1 -- 172.21.15.133:0/2049579518 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd8137c40 con 0x7fccd80b1820 2024-09-16T15:08:46.434 INFO:tasks.ceph.mgr.y.smithi191.stderr:2024-09-16T15:08:46.430+0000 7fcde9e98180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T15:08:46.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccbffff640 1 -- 172.21.15.133:0/2049579518 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 12) v1 ==== 128979+0+0 (secure 0 0 0) 0x7fcccc0b2040 con 0x7fccd80b1820 2024-09-16T15:08:46.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccbffff640 20 mgrc handle_mgr_map mgrmap(e 12) v1 2024-09-16T15:08:46.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccbffff640 4 mgrc handle_mgr_map Got map version 12 2024-09-16T15:08:46.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccbffff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:46.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccbffff640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] 2024-09-16T15:08:46.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.428+0000 7fccbffff640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fccc4097960 0x7fccc4099e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:46.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.432+0000 7fccd77fe640 1 -- 172.21.15.133:0/2049579518 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fccc4097960 msgr2=0x7fccc4099e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.191:6824/164170957 2024-09-16T15:08:46.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.432+0000 7fccbffff640 1 -- 172.21.15.133:0/2049579518 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4825+0+0 (secure 0 0 0) 0x7fcccc15a6f0 con 0x7fccd80b1820 2024-09-16T15:08:46.435 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.432+0000 7fccd77fe640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fccc4097960 0x7fccc4099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T15:08:46.436 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.432+0000 7fccbffff640 1 -- 172.21.15.133:0/2049579518 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcccc127b30 con 0x7fccd80b1820 2024-09-16T15:08:46.443 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.438+0000 7f596badf180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T15:08:46.455 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.452+0000 7fccbffff640 1 -- 172.21.15.133:0/2049579518 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mgrmap(e 13) v1 ==== 128588+0+0 (secure 0 0 0) 0x7fcccc158f80 con 0x7fccd80b1820 2024-09-16T15:08:46.455 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.452+0000 7fccbffff640 20 mgrc handle_mgr_map mgrmap(e 13) v1 2024-09-16T15:08:46.456 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.452+0000 7fccbffff640 4 mgrc handle_mgr_map Got map version 13 2024-09-16T15:08:46.456 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.452+0000 7fccbffff640 4 mgrc handle_mgr_map Active mgr is now 2024-09-16T15:08:46.456 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.452+0000 7fccbffff640 4 mgrc reconnect Terminating session with v2:172.21.15.191:6824/164170957 2024-09-16T15:08:46.456 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.452+0000 7fccbffff640 1 -- 172.21.15.133:0/2049579518 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fccc4097960 msgr2=0x7fccc4099e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:08:46.456 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.452+0000 7fccbffff640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fccc4097960 0x7fccc4099e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.456 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.452+0000 7fccbffff640 4 mgrc reconnect No active mgr available yet 2024-09-16T15:08:46.526 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.522+0000 7f58f4855640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:46.526 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.522+0000 7f58f4855640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:46.526 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.522+0000 7f58f4855640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:46.526 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.522+0000 7f58f4855640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:46.527 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.522+0000 7f58f4855640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:46.527 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.522+0000 7f58f785b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:46.527 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.522+0000 7f58f785b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:46.527 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.522+0000 7f58f785b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:46.527 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.522+0000 7f58f785b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:46.527 INFO:tasks.ceph.mgr.z.smithi191.stderr:2024-09-16T15:08:46.522+0000 7f58f785b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:08:46.593 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.588+0000 7fccde0e1640 1 -- 172.21.15.133:0/2049579518 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/failure_domain"} v 0) v1 -- 0x7fccd8137c40 con 0x7fccd80b1820 2024-09-16T15:08:46.601 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.600+0000 7fccbffff640 1 -- 172.21.15.133:0/2049579518 <== mon.2 v2:172.21.15.133:3301/0 8 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/failure_domain"}]=0 v9) v1 ==== 111+0+0 (secure 0 0 0) 0x7fccd8137c40 con 0x7fccd80b1820 2024-09-16T15:08:46.688 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.684+0000 7fccde0e1640 1 -- 172.21.15.133:0/2049579518 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/failure_domain"} v 0) v1 -- 0x7fccd8137c40 con 0x7fccd80b1820 2024-09-16T15:08:46.689 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.684+0000 7fccbffff640 1 -- 172.21.15.133:0/2049579518 <== mon.2 v2:172.21.15.133:3301/0 9 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/localpool/failure_domain"}]=0 v9) v1 ==== 111+0+0 (secure 0 0 0) 0x7fccd8137c40 con 0x7fccd80b1820 2024-09-16T15:08:46.691 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 10 mgrc shutdown 2024-09-16T15:08:46.691 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 1 -- 172.21.15.133:0/2049579518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 msgr2=0x7fccd8137f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:46.691 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8137f20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcccc019f90 tx=0x7fcccc019100 comp rx=0 tx=0).stop 2024-09-16T15:08:46.691 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 1 -- 172.21.15.133:0/2049579518 shutdown_connections 2024-09-16T15:08:46.691 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.191:6824/164170957,v1:172.21.15.191:6825/164170957] conn(0x7fccc4097960 0x7fccc4099e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.691 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccd8141040 0x7fccd806e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.692 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fccd80b1820 0x7fccd8137f20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.692 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 1 --2- 172.21.15.133:0/2049579518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fccd8070210 0x7fccd806f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:46.692 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 1 -- 172.21.15.133:0/2049579518 >> 172.21.15.133:0/2049579518 conn(0x7fccd80b1e60 msgr2=0x7fccd80b2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:46.692 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 1 -- 172.21.15.133:0/2049579518 shutdown_connections 2024-09-16T15:08:46.692 INFO:tasks.workunit.client.0.smithi133.stderr:2024-09-16T15:08:46.688+0000 7fccde0e1640 1 -- 172.21.15.133:0/2049579518 wait complete. 2024-09-16T15:08:46.705 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo OK 2024-09-16T15:08:46.705 INFO:tasks.workunit.client.0.smithi133.stdout:OK 2024-09-16T15:08:46.706 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T15:08:46.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T15:08:46.762 INFO:tasks.workunit:Stopping ['mgr/test_localpool.sh'] on client.0... 2024-09-16T15:08:46.762 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T15:08:47.066 DEBUG:teuthology.parallel:result is None 2024-09-16T15:08:47.067 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T15:08:47.078 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T15:08:47.078 DEBUG:teuthology.orchestra.run.smithi133:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-16T15:08:47.136 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T15:08:47.136 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T15:08:47.145 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T15:08:47.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.296+0000 7f510ea92640 1 Processor -- start 2024-09-16T15:08:47.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510ea92640 1 -- start start 2024-09-16T15:08:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510ea92640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f51080a9ae0 0x7f51080a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510ea92640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51080ab810 0x7f51080a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510ea92640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5108139b60 con 0x7f51081365b0 2024-09-16T15:08:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510ea92640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51081399f0 con 0x7f51080ab810 2024-09-16T15:08:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510ea92640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51080a5ca0 con 0x7f51080a9ae0 2024-09-16T15:08:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510d02f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51080ab810 0x7f51080a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510d02f640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51080ab810 0x7f51080a8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33286/0 (socket says 172.21.15.133:33286) 2024-09-16T15:08:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510d02f640 1 -- 172.21.15.133:0/4230594189 learned_addr learned my addr 172.21.15.133:0/4230594189 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f5107fff640 1 --2- 172.21.15.133:0/4230594189 >> v2:172.21.15.191:3300/0 conn(0x7f51080a9ae0 0x7f51080a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510d02f640 1 -- 172.21.15.133:0/4230594189 >> v2:172.21.15.191:3300/0 conn(0x7f51080a9ae0 msgr2=0x7f51080a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510d02f640 1 --2- 172.21.15.133:0/4230594189 >> v2:172.21.15.191:3300/0 conn(0x7f51080a9ae0 0x7f51080a80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510d02f640 1 -- 172.21.15.133:0/4230594189 >> v1:172.21.15.133:6789/0 conn(0x7f51081365b0 legacy=0x7f51080aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.300+0000 7f510d02f640 1 -- 172.21.15.133:0/4230594189 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5108136990 con 0x7f51080ab810 2024-09-16T15:08:47.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510d02f640 1 --2- 172.21.15.133:0/4230594189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080ab810 0x7f51080a8610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f51080aa5f0 tx=0x7f510001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bf2e0e13d53373 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:47.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f51077fe640 1 -- 172.21.15.133:0/4230594189 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000a5570 con 0x7f51080ab810 2024-09-16T15:08:47.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f51077fe640 1 -- 172.21.15.133:0/4230594189 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51000a56d0 con 0x7f51080ab810 2024-09-16T15:08:47.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f51077fe640 1 -- 172.21.15.133:0/4230594189 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000a5a60 con 0x7f51080ab810 2024-09-16T15:08:47.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- 172.21.15.133:0/4230594189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080ab810 msgr2=0x7f51080a8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 --2- 172.21.15.133:0/4230594189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080ab810 0x7f51080a8610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f51080aa5f0 tx=0x7f510001c8b0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- 172.21.15.133:0/4230594189 shutdown_connections 2024-09-16T15:08:47.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 --2- 172.21.15.133:0/4230594189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51080ab810 0x7f51080a8610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 --2- 172.21.15.133:0/4230594189 >> v2:172.21.15.191:3300/0 conn(0x7f51080a9ae0 0x7f51080a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- 172.21.15.133:0/4230594189 >> 172.21.15.133:0/4230594189 conn(0x7f510806a030 msgr2=0x7f510806bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:47.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- 172.21.15.133:0/4230594189 shutdown_connections 2024-09-16T15:08:47.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- 172.21.15.133:0/4230594189 wait complete. 2024-09-16T15:08:47.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 Processor -- start 2024-09-16T15:08:47.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- start start 2024-09-16T15:08:47.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51080a9ae0 0x7f5108130f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51080ab810 0x7f510812fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51081365b0 0x7f51081df1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081dbc60 con 0x7f51081365b0 2024-09-16T15:08:47.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5108131750 con 0x7f51080a9ae0 2024-09-16T15:08:47.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5108130a30 con 0x7f51080ab810 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f5107fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51080ab810 0x7f510812fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510d02f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51081365b0 0x7f51081df1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f5107fff640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51080ab810 0x7f510812fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55052/0 (socket says 172.21.15.133:55052) 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f5107fff640 1 -- 172.21.15.133:0/2738193992 learned_addr learned my addr 172.21.15.133:0/2738193992 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510c82e640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51080a9ae0 0x7f5108130f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f5107fff640 1 -- 172.21.15.133:0/2738193992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51080a9ae0 msgr2=0x7f5108130f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f5107fff640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51080a9ae0 0x7f5108130f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f5107fff640 1 -- 172.21.15.133:0/2738193992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51081365b0 msgr2=0x7f51081df1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f5107fff640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51081365b0 0x7f51081df1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f5107fff640 1 -- 172.21.15.133:0/2738193992 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100009b30 con 0x7f51080ab810 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510d02f640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51081365b0 0x7f51081df1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:47.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f5107fff640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51080ab810 0x7f510812fba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00d800 tx=0x7f50fc00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:47.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f51057fa640 1 -- 172.21.15.133:0/2738193992 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00b6c0 con 0x7f51080ab810 2024-09-16T15:08:47.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f51057fa640 1 -- 172.21.15.133:0/2738193992 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc00b820 con 0x7f51080ab810 2024-09-16T15:08:47.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- 172.21.15.133:0/2738193992 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5108131470 con 0x7f51080ab810 2024-09-16T15:08:47.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f51057fa640 1 -- 172.21.15.133:0/2738193992 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc00bb90 con 0x7f51080ab810 2024-09-16T15:08:47.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.304+0000 7f510ea92640 1 -- 172.21.15.133:0/2738193992 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081356b0 con 0x7f51080ab810 2024-09-16T15:08:47.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.308+0000 7f510ea92640 1 -- 172.21.15.133:0/2738193992 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51080672a0 con 0x7f51080ab810 2024-09-16T15:08:47.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.308+0000 7f51057fa640 1 -- 172.21.15.133:0/2738193992 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 13) v1 ==== 128588+0+0 (secure 0 0 0) 0x7f50fc00bcf0 con 0x7f51080ab810 2024-09-16T15:08:47.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.308+0000 7f51057fa640 20 mgrc handle_mgr_map mgrmap(e 13) v1 2024-09-16T15:08:47.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.308+0000 7f51057fa640 4 mgrc handle_mgr_map Got map version 13 2024-09-16T15:08:47.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.308+0000 7f51057fa640 4 mgrc handle_mgr_map Active mgr is now 2024-09-16T15:08:47.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.308+0000 7f51057fa640 4 mgrc reconnect No active mgr available yet 2024-09-16T15:08:47.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.308+0000 7f51057fa640 1 -- 172.21.15.133:0/2738193992 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5169+0+0 (secure 0 0 0) 0x7f50fc0c0a30 con 0x7f51080ab810 2024-09-16T15:08:47.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.312+0000 7f51057fa640 1 -- 172.21.15.133:0/2738193992 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f50fc08e260 con 0x7f51080ab810 2024-09-16T15:08:47.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.468+0000 7f51057fa640 1 -- 172.21.15.133:0/2738193992 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mgrmap(e 14) v1 ==== 128715+0+0 (secure 0 0 0) 0x7f50fc08dd10 con 0x7f51080ab810 2024-09-16T15:08:47.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.468+0000 7f51057fa640 20 mgrc handle_mgr_map mgrmap(e 14) v1 2024-09-16T15:08:47.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.468+0000 7f51057fa640 4 mgrc handle_mgr_map Got map version 14 2024-09-16T15:08:47.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.468+0000 7f51057fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:47.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.468+0000 7f51057fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:47.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.468+0000 7f51057fa640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f50f409cbf0 0x7f50f409efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.472+0000 7f510c82e640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f50f409cbf0 0x7f50f409efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.476+0000 7f510c82e640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f50f409cbf0 0x7f50f409efe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f50f8000f70 tx=0x7f50f8009040 comp rx=0 tx=0).ready entity=mgr.4504 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:47.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.484+0000 7f510ea92640 1 -- 172.21.15.133:0/2738193992 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f510814d0a0 con 0x7f51080ab810 2024-09-16T15:08:47.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.496+0000 7f51057fa640 1 -- 172.21.15.133:0/2738193992 <== mon.2 v2:172.21.15.133:3301/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f50fc086380 con 0x7f51080ab810 2024-09-16T15:08:47.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.496+0000 7f51057fa640 1 -- 172.21.15.133:0/2738193992 <== mon.2 v2:172.21.15.133:3301/0 9 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f510814d0a0 con 0x7f51080ab810 2024-09-16T15:08:47.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 10 mgrc shutdown 2024-09-16T15:08:47.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 -- 172.21.15.133:0/2738193992 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f50f409cbf0 msgr2=0x7f50f409efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f50f409cbf0 0x7f50f409efe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f50f8000f70 tx=0x7f50f8009040 comp rx=0 tx=0).stop 2024-09-16T15:08:47.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 -- 172.21.15.133:0/2738193992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51080ab810 msgr2=0x7f510812fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51080ab810 0x7f510812fba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00d800 tx=0x7f50fc00dcc0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 -- 172.21.15.133:0/2738193992 shutdown_connections 2024-09-16T15:08:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f50f409cbf0 0x7f50f409efe0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51081365b0 0x7f51081df1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51080ab810 0x7f510812fba0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 --2- 172.21.15.133:0/2738193992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51080a9ae0 0x7f5108130f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 -- 172.21.15.133:0/2738193992 >> 172.21.15.133:0/2738193992 conn(0x7f510806a030 msgr2=0x7f51080b8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 -- 172.21.15.133:0/2738193992 shutdown_connections 2024-09-16T15:08:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.500+0000 7f510ea92640 1 -- 172.21.15.133:0/2738193992 wait complete. 2024-09-16T15:08:47.519 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:08:47.520 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:08:47.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.628+0000 7f64875b4640 1 Processor -- start 2024-09-16T15:08:47.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.628+0000 7f64875b4640 1 -- start start 2024-09-16T15:08:47.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f64875b4640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f648009e490 0x7f648006afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f64875b4640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f648006a8f0 con 0x7f64800a1010 2024-09-16T15:08:47.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f64875b4640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64800a5bd0 con 0x7f64800a2cf0 2024-09-16T15:08:47.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f64875b4640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64800a4cf0 con 0x7f648009e490 2024-09-16T15:08:47.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6485b51640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f64800a1010 0x7f648014bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43506/0 (socket says 172.21.15.133:43506) 2024-09-16T15:08:47.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6485b51640 1 -- 172.21.15.133:0/1427921314 learned_addr learned my addr 172.21.15.133:0/1427921314 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:08:47.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848441143 0 0) 0x7f648006a8f0 con 0x7f64800a1010 2024-09-16T15:08:47.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f646c003530 con 0x7f64800a1010 2024-09-16T15:08:47.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096189110 0 0) 0x7f646c003530 con 0x7f64800a1010 2024-09-16T15:08:47.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 >> v2:172.21.15.191:3300/0 conn(0x7f648009e490 msgr2=0x7f648006afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 --2- 172.21.15.133:0/1427921314 >> v2:172.21.15.191:3300/0 conn(0x7f648009e490 0x7f648006afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 >> v1:172.21.15.191:6789/0 conn(0x7f64800a2cf0 legacy=0x7f64800a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64800a3950 con 0x7f64800a1010 2024-09-16T15:08:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f6478001a00 con 0x7f64800a1010 2024-09-16T15:08:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 >> v1:172.21.15.133:6789/0 conn(0x7f64800a1010 legacy=0x7f648014bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f646c003980 0x7f646c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f646c0047d0 0x7f646c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646c005390 0x7f646c00b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64800a4cf0 con 0x7f646c0047d0 2024-09-16T15:08:47.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c00c2d0 con 0x7f646c005390 2024-09-16T15:08:47.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c00c490 con 0x7f646c003980 2024-09-16T15:08:47.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6485350640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f646c003980 0x7f646c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6484b4f640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f646c0047d0 0x7f646c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6485350640 1 -- 172.21.15.133:0/1427921314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646c005390 msgr2=0x7f646c00b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:47.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6485350640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646c005390 0x7f646c00b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6485350640 1 -- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f646c0047d0 msgr2=0x7f646c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:47.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6485350640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f646c0047d0 0x7f646c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6485350640 1 -- 172.21.15.133:0/1427921314 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c003530 con 0x7f646c003980 2024-09-16T15:08:47.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6485b51640 1 -- 172.21.15.133:0/1427921314 reap_dead start 2024-09-16T15:08:47.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6484b4f640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f646c0047d0 0x7f646c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:47.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6485350640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f646c003980 0x7f646c003d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f647c001650 tx=0x7f647c01c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44243ce6bda4da9f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:47.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c005270 con 0x7f646c003980 2024-09-16T15:08:47.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f647c0a7070 con 0x7f646c003980 2024-09-16T15:08:47.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.632+0000 7f6477fff640 1 -- 172.21.15.133:0/1427921314 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c003620 con 0x7f646c003980 2024-09-16T15:08:47.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f646c003980 msgr2=0x7f646c003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f646c003980 0x7f646c003d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f647c001650 tx=0x7f647c01c2e0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- 172.21.15.133:0/1427921314 shutdown_connections 2024-09-16T15:08:47.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 --2- 172.21.15.133:0/1427921314 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f646c003980 0x7f646c003d80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- 172.21.15.133:0/1427921314 >> 172.21.15.133:0/1427921314 conn(0x7f64800a3fd0 msgr2=0x7f64800a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:47.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- 172.21.15.133:0/1427921314 shutdown_connections 2024-09-16T15:08:47.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- 172.21.15.133:0/1427921314 wait complete. 2024-09-16T15:08:47.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 Processor -- start 2024-09-16T15:08:47.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- start start 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646c003980 0x7f64801cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64801ceb50 0x7f64801cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f648013b7a0 0x7f648013bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64801cf650 con 0x7f648013b7a0 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64801cf910 con 0x7f646c003980 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64801ce630 con 0x7f64801ceb50 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6484b4f640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64801ceb50 0x7f64801cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6485b51640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f648013b7a0 0x7f648013bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6484b4f640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64801ceb50 0x7f64801cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55076/0 (socket says 172.21.15.133:55076) 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6484b4f640 1 -- 172.21.15.133:0/429594182 learned_addr learned my addr 172.21.15.133:0/429594182 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:47.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6485350640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646c003980 0x7f64801cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6484b4f640 1 -- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646c003980 msgr2=0x7f64801cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6484b4f640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646c003980 0x7f64801cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6484b4f640 1 -- 172.21.15.133:0/429594182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f648013b7a0 msgr2=0x7f648013bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6484b4f640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f648013b7a0 0x7f648013bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6484b4f640 1 -- 172.21.15.133:0/429594182 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648013c4a0 con 0x7f64801ceb50 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6485350640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646c003980 0x7f64801cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6484b4f640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64801ceb50 0x7f64801cef90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6470009fd0 tx=0x7f647000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6475ffb640 1 -- 172.21.15.133:0/429594182 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647000eec0 con 0x7f64801ceb50 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6475ffb640 1 -- 172.21.15.133:0/429594182 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f647000c940 con 0x7f64801ceb50 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- 172.21.15.133:0/429594182 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648006fb40 con 0x7f64801ceb50 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f6475ffb640 1 -- 172.21.15.133:0/429594182 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647000ccf0 con 0x7f64801ceb50 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- 172.21.15.133:0/429594182 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64800a4490 con 0x7f64801ceb50 2024-09-16T15:08:47.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.636+0000 7f64875b4640 1 -- 172.21.15.133:0/429594182 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64801cd960 con 0x7f64801ceb50 2024-09-16T15:08:47.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.640+0000 7f6475ffb640 1 -- 172.21.15.133:0/429594182 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 14) v1 ==== 128715+0+0 (secure 0 0 0) 0x7f6470013470 con 0x7f64801ceb50 2024-09-16T15:08:47.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.640+0000 7f6475ffb640 20 mgrc handle_mgr_map mgrmap(e 14) v1 2024-09-16T15:08:47.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.640+0000 7f6475ffb640 4 mgrc handle_mgr_map Got map version 14 2024-09-16T15:08:47.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.640+0000 7f6475ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:47.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.640+0000 7f6475ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:47.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.640+0000 7f6475ffb640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f6464097690 0x7f6464099b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.640+0000 7f6475ffb640 1 -- 172.21.15.133:0/429594182 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5169+0+0 (secure 0 0 0) 0x7f64700c0320 con 0x7f64801ceb50 2024-09-16T15:08:47.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.640+0000 7f6485350640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f6464097690 0x7f6464099b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.640+0000 7f6485350640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f6464097690 0x7f6464099b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f647c0aa000 tx=0x7f647c001030 comp rx=0 tx=0).ready entity=mgr.4504 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:47.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.644+0000 7f6475ffb640 1 -- 172.21.15.133:0/429594182 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f647008da40 con 0x7f64801ceb50 2024-09-16T15:08:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 -- 172.21.15.133:0/429594182 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f648006ba00 con 0x7f64801ceb50 2024-09-16T15:08:47.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f6475ffb640 1 -- 172.21.15.133:0/429594182 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3978 (secure 0 0 0) 0x7f6470017090 con 0x7f64801ceb50 2024-09-16T15:08:47.843 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:08:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6811/3104644688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3104644688},{"type":"v1","addr":"172.21.15.133:6811","nonce":3104644688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6809/147821810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":147821810},{"type":"v1","addr":"172.21.15.133:6809","nonce":147821810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:07:22.048013+0000","modified":"2024-09-16T15:07:24.720576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6827/2751563681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2751563681},{"type":"v1","addr":"172.21.15.191:6827","nonce":2751563681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-16T15:08:47.843 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-09-16T15:08:47.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 10 mgrc shutdown 2024-09-16T15:08:47.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 -- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f6464097690 msgr2=0x7f6464099b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f6464097690 0x7f6464099b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f647c0aa000 tx=0x7f647c001030 comp rx=0 tx=0).stop 2024-09-16T15:08:47.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 -- 172.21.15.133:0/429594182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64801ceb50 msgr2=0x7f64801cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64801ceb50 0x7f64801cef90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6470009fd0 tx=0x7f647000ec70 comp rx=0 tx=0).stop 2024-09-16T15:08:47.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 -- 172.21.15.133:0/429594182 shutdown_connections 2024-09-16T15:08:47.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f6464097690 0x7f6464099b50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f648013b7a0 0x7f648013bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64801ceb50 0x7f64801cef90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 --2- 172.21.15.133:0/429594182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f646c003980 0x7f64801cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 -- 172.21.15.133:0/429594182 >> 172.21.15.133:0/429594182 conn(0x7f64800a3fd0 msgr2=0x7f64800b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:47.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 -- 172.21.15.133:0/429594182 shutdown_connections 2024-09-16T15:08:47.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.840+0000 7f64875b4640 1 -- 172.21.15.133:0/429594182 wait complete. 2024-09-16T15:08:47.863 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:08:47.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853d1b8640 1 Processor -- start 2024-09-16T15:08:47.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853d1b8640 1 -- start start 2024-09-16T15:08:47.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853d1b8640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f85380a2cf0 0x7f85380a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853d1b8640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85380a1010 0x7f853814bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853d1b8640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853806a8f0 con 0x7f853809e490 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853d1b8640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85380a5bd0 con 0x7f85380a2cf0 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853d1b8640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85380a5910 con 0x7f85380a1010 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f8536d9d640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f85380a2cf0 0x7f85380a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853759e640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85380a1010 0x7f853814bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853659c640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f853809e490 0x7f853806afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43508/0 (socket says 172.21.15.133:43508) 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853659c640 1 -- 172.21.15.133:0/3893636404 learned_addr learned my addr 172.21.15.133:0/3893636404 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f8536d9d640 1 -- 172.21.15.133:0/3893636404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85380a1010 msgr2=0x7f853814bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f8536d9d640 1 --2- 172.21.15.133:0/3893636404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85380a1010 0x7f853814bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f8536d9d640 1 -- 172.21.15.133:0/3893636404 >> v1:172.21.15.133:6789/0 conn(0x7f853809e490 legacy=0x7f853806afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f8535d9b640 1 -- 172.21.15.133:0/3893636404 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885141266 0 0) 0x7f853806a8f0 con 0x7f853809e490 2024-09-16T15:08:47.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f8536d9d640 1 -- 172.21.15.133:0/3893636404 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85380a4cf0 con 0x7f85380a2cf0 2024-09-16T15:08:47.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.980+0000 7f853759e640 1 --2- 172.21.15.133:0/3893636404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85380a1010 0x7f853814bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:47.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8536d9d640 1 --2- 172.21.15.133:0/3893636404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85380a2cf0 0x7f85380a53d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f85280097c0 tx=0x7f852801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46da78fe471242b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:47.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8535d9b640 1 -- 172.21.15.133:0/3893636404 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280ad660 con 0x7f85380a2cf0 2024-09-16T15:08:47.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8535d9b640 1 -- 172.21.15.133:0/3893636404 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f85280ad7c0 con 0x7f85380a2cf0 2024-09-16T15:08:47.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8535d9b640 1 -- 172.21.15.133:0/3893636404 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280adb70 con 0x7f85380a2cf0 2024-09-16T15:08:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 -- 172.21.15.133:0/3893636404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85380a2cf0 msgr2=0x7f85380a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 --2- 172.21.15.133:0/3893636404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85380a2cf0 0x7f85380a53d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f85280097c0 tx=0x7f852801c360 comp rx=0 tx=0).stop 2024-09-16T15:08:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 -- 172.21.15.133:0/3893636404 shutdown_connections 2024-09-16T15:08:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 --2- 172.21.15.133:0/3893636404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85380a1010 0x7f853814bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 --2- 172.21.15.133:0/3893636404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85380a2cf0 0x7f85380a53d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 -- 172.21.15.133:0/3893636404 >> 172.21.15.133:0/3893636404 conn(0x7f85380a3fd0 msgr2=0x7f85380a43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 -- 172.21.15.133:0/3893636404 shutdown_connections 2024-09-16T15:08:47.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 -- 172.21.15.133:0/3893636404 wait complete. 2024-09-16T15:08:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 Processor -- start 2024-09-16T15:08:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 -- start start 2024-09-16T15:08:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f853809e490 0x7f85381d9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85380a1010 0x7f8538099120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85380a2cf0 0x7f85381419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8538099660 con 0x7f85380a1010 2024-09-16T15:08:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85381da6e0 con 0x7f85380a2cf0 2024-09-16T15:08:47.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853d1b8640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85381da910 con 0x7f853809e490 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f853659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85380a1010 0x7f8538099120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8536d9d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f853809e490 0x7f85381d9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8536d9d640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f853809e490 0x7f85381d9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55084/0 (socket says 172.21.15.133:55084) 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8536d9d640 1 -- 172.21.15.133:0/570142538 learned_addr learned my addr 172.21.15.133:0/570142538 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8536d9d640 1 -- 172.21.15.133:0/570142538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85380a2cf0 msgr2=0x7f85381419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8536d9d640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85380a2cf0 0x7f85381419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8536d9d640 1 -- 172.21.15.133:0/570142538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85380a1010 msgr2=0x7f8538099120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8536d9d640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85380a1010 0x7f8538099120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.984+0000 7f8536d9d640 1 -- 172.21.15.133:0/570142538 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530009a70 con 0x7f853809e490 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f853659c640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85380a1010 0x7f8538099120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f8536d9d640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f853809e490 0x7f85381d9230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f852801aa10 tx=0x7f8528019100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f851f7fe640 1 -- 172.21.15.133:0/570142538 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280b6610 con 0x7f853809e490 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f851f7fe640 1 -- 172.21.15.133:0/570142538 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f85280b6770 con 0x7f853809e490 2024-09-16T15:08:47.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f851f7fe640 1 -- 172.21.15.133:0/570142538 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280b6b40 con 0x7f853809e490 2024-09-16T15:08:47.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f853d1b8640 1 -- 172.21.15.133:0/570142538 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8528009420 con 0x7f853809e490 2024-09-16T15:08:47.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f853d1b8640 1 -- 172.21.15.133:0/570142538 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853806b870 con 0x7f853809e490 2024-09-16T15:08:47.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f853d1b8640 1 -- 172.21.15.133:0/570142538 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85380a53d0 con 0x7f853809e490 2024-09-16T15:08:47.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f851f7fe640 1 -- 172.21.15.133:0/570142538 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 14) v1 ==== 128715+0+0 (secure 0 0 0) 0x7f85280b4040 con 0x7f853809e490 2024-09-16T15:08:47.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f851f7fe640 20 mgrc handle_mgr_map mgrmap(e 14) v1 2024-09-16T15:08:47.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f851f7fe640 4 mgrc handle_mgr_map Got map version 14 2024-09-16T15:08:47.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f851f7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:47.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f851f7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:47.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f851f7fe640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f8524097690 0x7f8524099b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:47.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f851f7fe640 1 -- 172.21.15.133:0/570142538 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5169+0+0 (secure 0 0 0) 0x7f852815a2b0 con 0x7f853809e490 2024-09-16T15:08:47.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.988+0000 7f853659c640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f8524097690 0x7f8524099b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:47.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.992+0000 7f853659c640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f8524097690 0x7f8524099b50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f852c007fd0 tx=0x7f852c007d60 comp rx=0 tx=0).ready entity=mgr.4504 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:47.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:47.992+0000 7f851f7fe640 1 -- 172.21.15.133:0/570142538 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f85281279d0 con 0x7f853809e490 2024-09-16T15:08:48.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 -- 172.21.15.133:0/570142538 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85381d9770 con 0x7f853809e490 2024-09-16T15:08:48.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f851f7fe640 1 -- 172.21.15.133:0/570142538 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+3978 (secure 0 0 0) 0x7f8528129900 con 0x7f853809e490 2024-09-16T15:08:48.191 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:08:48.191 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4193,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6811/3104644688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3104644688},{"type":"v1","addr":"172.21.15.133:6811","nonce":3104644688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6809/147821810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":147821810},{"type":"v1","addr":"172.21.15.133:6809","nonce":147821810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:07:22.048013+0000","modified":"2024-09-16T15:07:24.720576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6827/2751563681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2751563681},{"type":"v1","addr":"172.21.15.191:6827","nonce":2751563681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-16T15:08:48.192 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2024-09-16T15:08:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 10 mgrc shutdown 2024-09-16T15:08:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 -- 172.21.15.133:0/570142538 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f8524097690 msgr2=0x7f8524099b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f8524097690 0x7f8524099b50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f852c007fd0 tx=0x7f852c007d60 comp rx=0 tx=0).stop 2024-09-16T15:08:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 -- 172.21.15.133:0/570142538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f853809e490 msgr2=0x7f85381d9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f853809e490 0x7f85381d9230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f852801aa10 tx=0x7f8528019100 comp rx=0 tx=0).stop 2024-09-16T15:08:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 -- 172.21.15.133:0/570142538 shutdown_connections 2024-09-16T15:08:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f8524097690 0x7f8524099b50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85380a2cf0 0x7f85381419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85380a1010 0x7f8538099120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 --2- 172.21.15.133:0/570142538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f853809e490 0x7f85381d9230 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 -- 172.21.15.133:0/570142538 >> 172.21.15.133:0/570142538 conn(0x7f85380a3fd0 msgr2=0x7f85380b4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 -- 172.21.15.133:0/570142538 shutdown_connections 2024-09-16T15:08:48.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.188+0000 7f853d1b8640 1 -- 172.21.15.133:0/570142538 wait complete. 2024-09-16T15:08:48.210 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:08:48.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.324+0000 7f14e8748640 1 Processor -- start 2024-09-16T15:08:48.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.324+0000 7f14e8748640 1 -- start start 2024-09-16T15:08:48.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.324+0000 7f14e8748640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f14e01365b0 0x7f14e00aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.324+0000 7f14e8748640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e0139b60 con 0x7f14e00a9ae0 2024-09-16T15:08:48.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.324+0000 7f14e8748640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e00a5ca0 con 0x7f14e00ab810 2024-09-16T15:08:48.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.324+0000 7f14e8748640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e0136990 con 0x7f14e01365b0 2024-09-16T15:08:48.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.324+0000 7f14e64e4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f14e01365b0 0x7f14e00aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e64e4640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f14e01365b0 0x7f14e00aa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33322/0 (socket says 172.21.15.133:33322) 2024-09-16T15:08:48.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e64e4640 1 -- 172.21.15.133:0/3805647624 learned_addr learned my addr 172.21.15.133:0/3805647624 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:48.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.324+0000 7f14e5ce3640 1 --1- 172.21.15.133:0/3805647624 >> v1:172.21.15.133:6789/0 conn(0x7f14e00a9ae0 0x7f14e00a80d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43520/0 (socket says 172.21.15.133:43520) 2024-09-16T15:08:48.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e64e4640 1 -- 172.21.15.133:0/3805647624 >> v1:172.21.15.191:6789/0 conn(0x7f14e00ab810 legacy=0x7f14e014dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:48.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e64e4640 1 -- 172.21.15.133:0/3805647624 >> v1:172.21.15.133:6789/0 conn(0x7f14e00a9ae0 legacy=0x7f14e00a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e54e2640 1 -- 172.21.15.133:0/3805647624 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050846300 0 0) 0x7f14e0139b60 con 0x7f14e00a9ae0 2024-09-16T15:08:48.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e64e4640 1 -- 172.21.15.133:0/3805647624 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e00a6a60 con 0x7f14e01365b0 2024-09-16T15:08:48.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e64e4640 1 --2- 172.21.15.133:0/3805647624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 0x7f14e00aa960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009870 tx=0x7f14dc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a5f199ac9d938 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:48.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e54e2640 1 -- 172.21.15.133:0/3805647624 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0ab600 con 0x7f14e01365b0 2024-09-16T15:08:48.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e54e2640 1 -- 172.21.15.133:0/3805647624 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f14dc0ab760 con 0x7f14e01365b0 2024-09-16T15:08:48.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e54e2640 1 -- 172.21.15.133:0/3805647624 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0abb10 con 0x7f14e01365b0 2024-09-16T15:08:48.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e8748640 1 -- 172.21.15.133:0/3805647624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 msgr2=0x7f14e00aa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e8748640 1 --2- 172.21.15.133:0/3805647624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 0x7f14e00aa960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009870 tx=0x7f14dc01c3e0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e8748640 1 -- 172.21.15.133:0/3805647624 shutdown_connections 2024-09-16T15:08:48.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e8748640 1 --2- 172.21.15.133:0/3805647624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 0x7f14e00aa960 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e8748640 1 -- 172.21.15.133:0/3805647624 >> 172.21.15.133:0/3805647624 conn(0x7f14e006a030 msgr2=0x7f14e006bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:48.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e8748640 1 -- 172.21.15.133:0/3805647624 shutdown_connections 2024-09-16T15:08:48.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.328+0000 7f14e8748640 1 -- 172.21.15.133:0/3805647624 wait complete. 2024-09-16T15:08:48.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 Processor -- start 2024-09-16T15:08:48.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 -- start start 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14e00a9ae0 0x7f14e0147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14e00ab810 0x7f14e0149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 0x7f14e00a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e0148310 con 0x7f14e01365b0 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00a5470 con 0x7f14e00ab810 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e0133be0 con 0x7f14e00a9ae0 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e6ce5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 0x7f14e00a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e64e4640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14e00a9ae0 0x7f14e0147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e6ce5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 0x7f14e00a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33326/0 (socket says 172.21.15.133:33326) 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e6ce5640 1 -- 172.21.15.133:0/2301236511 learned_addr learned my addr 172.21.15.133:0/2301236511 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e6ce5640 1 -- 172.21.15.133:0/2301236511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14e00a9ae0 msgr2=0x7f14e0147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e6ce5640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14e00a9ae0 0x7f14e0147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e5ce3640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14e00ab810 0x7f14e0149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e6ce5640 1 -- 172.21.15.133:0/2301236511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14e00ab810 msgr2=0x7f14e0149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e6ce5640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14e00ab810 0x7f14e0149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e6ce5640 1 -- 172.21.15.133:0/2301236511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc0094d0 con 0x7f14e01365b0 2024-09-16T15:08:48.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e6ce5640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 0x7f14e00a4270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c780 tx=0x7f14d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:48.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14d6ffd640 1 -- 172.21.15.133:0/2301236511 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d800a9f0 con 0x7f14e01365b0 2024-09-16T15:08:48.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14d6ffd640 1 -- 172.21.15.133:0/2301236511 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f14d800ce90 con 0x7f14e01365b0 2024-09-16T15:08:48.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 -- 172.21.15.133:0/2301236511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e0133e50 con 0x7f14e01365b0 2024-09-16T15:08:48.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14d6ffd640 1 -- 172.21.15.133:0/2301236511 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d800adf0 con 0x7f14e01365b0 2024-09-16T15:08:48.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 -- 172.21.15.133:0/2301236511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e0134670 con 0x7f14e01365b0 2024-09-16T15:08:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.332+0000 7f14e8748640 1 -- 172.21.15.133:0/2301236511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e00672a0 con 0x7f14e01365b0 2024-09-16T15:08:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.336+0000 7f14d6ffd640 1 -- 172.21.15.133:0/2301236511 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 14) v1 ==== 128715+0+0 (secure 0 0 0) 0x7f14d8024430 con 0x7f14e01365b0 2024-09-16T15:08:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.336+0000 7f14d6ffd640 20 mgrc handle_mgr_map mgrmap(e 14) v1 2024-09-16T15:08:48.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.336+0000 7f14d6ffd640 4 mgrc handle_mgr_map Got map version 14 2024-09-16T15:08:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.336+0000 7f14d6ffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.336+0000 7f14d6ffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.336+0000 7f14d6ffd640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f14c4097690 0x7f14c4099b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.336+0000 7f14d6ffd640 1 -- 172.21.15.133:0/2301236511 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5169+0+0 (secure 0 0 0) 0x7f14d80c02e0 con 0x7f14e01365b0 2024-09-16T15:08:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.336+0000 7f14e64e4640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f14c4097690 0x7f14c4099b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.336+0000 7f14e64e4640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f14c4097690 0x7f14c4099b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0af000 tx=0x7f14dc0b6040 comp rx=0 tx=0).ready entity=mgr.4504 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:48.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.340+0000 7f14d6ffd640 1 -- 172.21.15.133:0/2301236511 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f14d808da00 con 0x7f14e01365b0 2024-09-16T15:08:48.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.500+0000 7f14e8748640 1 -- 172.21.15.133:0/2301236511 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14e0133920 con 0x7f14e01365b0 2024-09-16T15:08:48.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14d6ffd640 1 -- 172.21.15.133:0/2301236511 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+14429 (secure 0 0 0) 0x7f14e0133920 con 0x7f14e01365b0 2024-09-16T15:08:48.506 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-16T15:08:48.507 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":33,"fsid":"9e097bec-4223-42be-ab5d-08197519f9c7","created":"2024-09-16T15:06:57.476632+0000","modified":"2024-09-16T15:08:46.446087+0000","last_up_change":"2024-09-16T15:07:07.565510+0000","last_in_change":"2024-09-16T15:06:59.549440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:07:08.758832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T15:07:14.143631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:07:20.523939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2024-09-16T15:07:21.043356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ce332fe7-4e7e-49ac-8a6b-efd5b6b4ddab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6801","nonce":191622751}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6803","nonce":191622751}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6807","nonce":191622751}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":191622751},{"type":"v1","addr":"172.21.15.133:6805","nonce":191622751}]},"public_addr":"172.21.15.133:6801/191622751","cluster_addr":"172.21.15.133:6803/191622751","heartbeat_back_addr":"172.21.15.133:6807/191622751","heartbeat_front_addr":"172.21.15.133:6805/191622751","state":["exists","up"]},{"osd":1,"uuid":"193fc20e-147a-4af2-9254-9239013f2784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6817","nonce":1482439914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6819","nonce":1482439914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6823","nonce":1482439914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1482439914},{"type":"v1","addr":"172.21.15.191:6821","nonce":1482439914}]},"public_addr":"172.21.15.191:6817/1482439914","cluster_addr":"172.21.15.191:6819/1482439914","heartbeat_back_addr":"172.21.15.191:6823/1482439914","heartbeat_front_addr":"172.21.15.191:6821/1482439914","state":["exists","up"]},{"osd":2,"uuid":"10507e21-41d0-4f66-ada7-6fb5a2837b77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6801","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6802","nonce":193816588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6803","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6805","nonce":193816588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6811","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6813","nonce":193816588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6807","nonce":193816588},{"type":"v1","addr":"172.21.15.191:6809","nonce":193816588}]},"public_addr":"172.21.15.191:6802/193816588","cluster_addr":"172.21.15.191:6805/193816588","heartbeat_back_addr":"172.21.15.191:6813/193816588","heartbeat_front_addr":"172.21.15.191:6809/193816588","state":["exists","up"]},{"osd":3,"uuid":"d4b8a403-1961-4a29-af2b-cfa07ce5f1bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6804","nonce":787707562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6808","nonce":787707562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6815","nonce":787707562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":787707562},{"type":"v1","addr":"172.21.15.191:6812","nonce":787707562}]},"public_addr":"172.21.15.191:6804/787707562","cluster_addr":"172.21.15.191:6808/787707562","heartbeat_back_addr":"172.21.15.191:6815/787707562","heartbeat_front_addr":"172.21.15.191:6812/787707562","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:04.712037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.140757+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.102351+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:07:05.172231+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.2d","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6825/164170957":"2024-09-17T15:08:46.446052+0000","172.21.15.191:0/3684856439":"2024-09-17T15:08:38.681344+0000","172.21.15.191:0/2465333633":"2024-09-17T15:08:38.681344+0000","172.21.15.191:0/1049131429":"2024-09-17T15:08:38.681344+0000","172.21.15.191:0/1928163351":"2024-09-17T15:08:38.681344+0000","172.21.15.191:6825/3926506186":"2024-09-17T15:08:38.681344+0000","172.21.15.191:0/1314335080":"2024-09-17T15:08:46.446052+0000","172.21.15.191:6824/3926506186":"2024-09-17T15:08:38.681344+0000","172.21.15.191:6824/164170957":"2024-09-17T15:08:46.446052+0000","172.21.15.191:0/956865301":"2024-09-17T15:08:46.446052+0000","172.21.15.191:0/210443317":"2024-09-17T15:08:46.446052+0000","172.21.15.191:0/734389274":"2024-09-17T15:08:46.446052+0000","172.21.15.191:0/1837400867":"2024-09-17T15:08:46.446052+0000","172.21.15.191:0/1051566210":"2024-09-17T15:08:46.446052+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:08:48.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 10 mgrc shutdown 2024-09-16T15:08:48.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 -- 172.21.15.133:0/2301236511 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f14c4097690 msgr2=0x7f14c4099b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f14c4097690 0x7f14c4099b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0af000 tx=0x7f14dc0b6040 comp rx=0 tx=0).stop 2024-09-16T15:08:48.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 -- 172.21.15.133:0/2301236511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 msgr2=0x7f14e00a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 0x7f14e00a4270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c780 tx=0x7f14d800cc40 comp rx=0 tx=0).stop 2024-09-16T15:08:48.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 -- 172.21.15.133:0/2301236511 shutdown_connections 2024-09-16T15:08:48.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f14c4097690 0x7f14c4099b50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14e01365b0 0x7f14e00a4270 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14e00ab810 0x7f14e0149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 --2- 172.21.15.133:0/2301236511 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14e00a9ae0 0x7f14e0147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 -- 172.21.15.133:0/2301236511 >> 172.21.15.133:0/2301236511 conn(0x7f14e006a030 msgr2=0x7f14e0071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:48.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 -- 172.21.15.133:0/2301236511 shutdown_connections 2024-09-16T15:08:48.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.504+0000 7f14e8748640 1 -- 172.21.15.133:0/2301236511 wait complete. 2024-09-16T15:08:48.526 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:08:48.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96b075f640 1 Processor -- start 2024-09-16T15:08:48.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96b075f640 1 -- start start 2024-09-16T15:08:48.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96b075f640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f96a81365b0 0x7f96a80aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96b075f640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96b075f640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a8139b60 con 0x7f96a80a9ae0 2024-09-16T15:08:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96b075f640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a80a5ca0 con 0x7f96a81365b0 2024-09-16T15:08:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96b075f640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8136990 con 0x7f96a80ab810 2024-09-16T15:08:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96aecfc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96aecfc640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a814dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55100/0 (socket says 172.21.15.133:55100) 2024-09-16T15:08:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96aecfc640 1 -- 172.21.15.133:0/1126580242 learned_addr learned my addr 172.21.15.133:0/1126580242 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96ae4fb640 1 --2- 172.21.15.133:0/1126580242 >> v2:172.21.15.191:3300/0 conn(0x7f96a81365b0 0x7f96a80aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96aecfc640 1 -- 172.21.15.133:0/1126580242 >> v2:172.21.15.191:3300/0 conn(0x7f96a81365b0 msgr2=0x7f96a80aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96aecfc640 1 --2- 172.21.15.133:0/1126580242 >> v2:172.21.15.191:3300/0 conn(0x7f96a81365b0 0x7f96a80aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96aecfc640 1 -- 172.21.15.133:0/1126580242 >> v1:172.21.15.191:6789/0 conn(0x7f96a80a9ae0 legacy=0x7f96a80a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:48.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.636+0000 7f96aecfc640 1 -- 172.21.15.133:0/1126580242 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80a6a60 con 0x7f96a80ab810 2024-09-16T15:08:48.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96aecfc640 1 --2- 172.21.15.133:0/1126580242 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a814dc40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009e50 tx=0x7f96a001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c30e2ef0a5d630 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:48.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96ad4f9640 1 -- 172.21.15.133:0/1126580242 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a00a5750 con 0x7f96a80ab810 2024-09-16T15:08:48.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96ad4f9640 1 -- 172.21.15.133:0/1126580242 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f96a00a58b0 con 0x7f96a80ab810 2024-09-16T15:08:48.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96ad4f9640 1 -- 172.21.15.133:0/1126580242 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a00a5c60 con 0x7f96a80ab810 2024-09-16T15:08:48.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 -- 172.21.15.133:0/1126580242 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 msgr2=0x7f96a814dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 --2- 172.21.15.133:0/1126580242 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a814dc40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009e50 tx=0x7f96a001c8b0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 -- 172.21.15.133:0/1126580242 shutdown_connections 2024-09-16T15:08:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 --2- 172.21.15.133:0/1126580242 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a814dc40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 --2- 172.21.15.133:0/1126580242 >> v2:172.21.15.191:3300/0 conn(0x7f96a81365b0 0x7f96a80aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 -- 172.21.15.133:0/1126580242 >> 172.21.15.133:0/1126580242 conn(0x7f96a806a030 msgr2=0x7f96a806bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 -- 172.21.15.133:0/1126580242 shutdown_connections 2024-09-16T15:08:48.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 -- 172.21.15.133:0/1126580242 wait complete. 2024-09-16T15:08:48.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 Processor -- start 2024-09-16T15:08:48.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 -- start start 2024-09-16T15:08:48.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a80a9ae0 0x7f96a8148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a81350b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a81365b0 0x7f96a809db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8149600 con 0x7f96a81365b0 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a81486f0 con 0x7f96a80a9ae0 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96b075f640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a81325f0 con 0x7f96a80ab810 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96adcfa640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a81350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96aecfc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a81365b0 0x7f96a809db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96adcfa640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a81350b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55114/0 (socket says 172.21.15.133:55114) 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96adcfa640 1 -- 172.21.15.133:0/2628000006 learned_addr learned my addr 172.21.15.133:0/2628000006 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96ae4fb640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a80a9ae0 0x7f96a8148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96adcfa640 1 -- 172.21.15.133:0/2628000006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a80a9ae0 msgr2=0x7f96a8148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96adcfa640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a80a9ae0 0x7f96a8148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96adcfa640 1 -- 172.21.15.133:0/2628000006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a81365b0 msgr2=0x7f96a809db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96adcfa640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a81365b0 0x7f96a809db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96adcfa640 1 -- 172.21.15.133:0/2628000006 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a0009b30 con 0x7f96a80ab810 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.640+0000 7f96aecfc640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a81365b0 0x7f96a809db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f96adcfa640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a81350b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c480 tx=0x7f96a400c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f969effd640 1 -- 172.21.15.133:0/2628000006 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4012830 con 0x7f96a80ab810 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f969effd640 1 -- 172.21.15.133:0/2628000006 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f96a40129b0 con 0x7f96a80ab810 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f96b075f640 1 -- 172.21.15.133:0/2628000006 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a81328b0 con 0x7f96a80ab810 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f969effd640 1 -- 172.21.15.133:0/2628000006 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4012d80 con 0x7f96a80ab810 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f96b075f640 1 -- 172.21.15.133:0/2628000006 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a8149130 con 0x7f96a80ab810 2024-09-16T15:08:48.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f96b075f640 1 -- 172.21.15.133:0/2628000006 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a80672a0 con 0x7f96a80ab810 2024-09-16T15:08:48.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f969effd640 1 -- 172.21.15.133:0/2628000006 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 14) v1 ==== 128715+0+0 (secure 0 0 0) 0x7f96a401c460 con 0x7f96a80ab810 2024-09-16T15:08:48.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f969effd640 20 mgrc handle_mgr_map mgrmap(e 14) v1 2024-09-16T15:08:48.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f969effd640 4 mgrc handle_mgr_map Got map version 14 2024-09-16T15:08:48.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f969effd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:48.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f969effd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:48.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f969effd640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f968c097690 0x7f968c099b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f969effd640 1 -- 172.21.15.133:0/2628000006 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5169+0+0 (secure 0 0 0) 0x7f96a40c06e0 con 0x7f96a80ab810 2024-09-16T15:08:48.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f96ae4fb640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f968c097690 0x7f968c099b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.644+0000 7f96ae4fb640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f968c097690 0x7f968c099b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f969800b930 tx=0x7f9698008040 comp rx=0 tx=0).ready entity=mgr.4504 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:48.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.648+0000 7f969effd640 1 -- 172.21.15.133:0/2628000006 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f96a408de00 con 0x7f96a80ab810 2024-09-16T15:08:48.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.844+0000 7f96b075f640 1 -- 172.21.15.133:0/2628000006 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f96a80aa960 con 0x7f96a80ab810 2024-09-16T15:08:48.867 INFO:tasks.ceph.mds.b.smithi191.stderr: -917> 2024-09-16T15:07:23.856+0000 7f4c13d01640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:08:48.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.864+0000 7f969effd640 1 -- 172.21.15.133:0/2628000006 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f96a408fd30 con 0x7f96a80ab810 2024-09-16T15:08:48.869 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:08:48.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 10 mgrc shutdown 2024-09-16T15:08:48.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 -- 172.21.15.133:0/2628000006 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f968c097690 msgr2=0x7f968c099b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f968c097690 0x7f968c099b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f969800b930 tx=0x7f9698008040 comp rx=0 tx=0).stop 2024-09-16T15:08:48.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 -- 172.21.15.133:0/2628000006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 msgr2=0x7f96a81350b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:48.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a81350b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c480 tx=0x7f96a400c940 comp rx=0 tx=0).stop 2024-09-16T15:08:48.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 -- 172.21.15.133:0/2628000006 shutdown_connections 2024-09-16T15:08:48.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f968c097690 0x7f968c099b50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96a81365b0 0x7f96a809db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96a80ab810 0x7f96a81350b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 --2- 172.21.15.133:0/2628000006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96a80a9ae0 0x7f96a8148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:48.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 -- 172.21.15.133:0/2628000006 >> 172.21.15.133:0/2628000006 conn(0x7f96a806a030 msgr2=0x7f96a80a6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:48.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 -- 172.21.15.133:0/2628000006 shutdown_connections 2024-09-16T15:08:48.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.868+0000 7f96b075f640 1 -- 172.21.15.133:0/2628000006 wait complete. 2024-09-16T15:08:48.888 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:08:48.912 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2024-09-16T15:08:48.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.992+0000 7f3a1c377640 1 Processor -- start 2024-09-16T15:08:48.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a1c377640 1 -- start start 2024-09-16T15:08:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a1c377640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3a140a9ae0 0x7f3a140a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a1c377640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 0x7f3a140a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a1c377640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a14139b60 con 0x7f3a141365b0 2024-09-16T15:08:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a1c377640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a141399f0 con 0x7f3a140a9ae0 2024-09-16T15:08:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a1c377640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140a5ca0 con 0x7f3a140ab810 2024-09-16T15:08:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19912640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3a140a9ae0 0x7f3a140a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19912640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3a140a9ae0 0x7f3a140a80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33346/0 (socket says 172.21.15.133:33346) 2024-09-16T15:08:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19912640 1 -- 172.21.15.133:0/3428255 learned_addr learned my addr 172.21.15.133:0/3428255 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:48.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a1a914640 1 --2- 172.21.15.133:0/3428255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 0x7f3a140a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:49.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19912640 1 -- 172.21.15.133:0/3428255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 msgr2=0x7f3a140a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:49.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19912640 1 --2- 172.21.15.133:0/3428255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 0x7f3a140a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:49.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19912640 1 -- 172.21.15.133:0/3428255 >> v1:172.21.15.191:6789/0 conn(0x7f3a141365b0 legacy=0x7f3a140aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:49.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19912640 1 -- 172.21.15.133:0/3428255 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a14136990 con 0x7f3a140a9ae0 2024-09-16T15:08:49.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19912640 1 --2- 172.21.15.133:0/3428255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a140a9ae0 0x7f3a140a80d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009870 tx=0x7f3a1001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7ec24990129342 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:49.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19111640 1 -- 172.21.15.133:0/3428255 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100ab600 con 0x7f3a140a9ae0 2024-09-16T15:08:49.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19111640 1 -- 172.21.15.133:0/3428255 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3a100a7050 con 0x7f3a140a9ae0 2024-09-16T15:08:49.001 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:48.996+0000 7f3a19111640 1 -- 172.21.15.133:0/3428255 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100ab9a0 con 0x7f3a140a9ae0 2024-09-16T15:08:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- 172.21.15.133:0/3428255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a140a9ae0 msgr2=0x7f3a140a80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 --2- 172.21.15.133:0/3428255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a140a9ae0 0x7f3a140a80d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009870 tx=0x7f3a1001c3e0 comp rx=0 tx=0).stop 2024-09-16T15:08:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- 172.21.15.133:0/3428255 shutdown_connections 2024-09-16T15:08:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 --2- 172.21.15.133:0/3428255 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 0x7f3a140a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:49.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 --2- 172.21.15.133:0/3428255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a140a9ae0 0x7f3a140a80d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:49.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- 172.21.15.133:0/3428255 >> 172.21.15.133:0/3428255 conn(0x7f3a1406a030 msgr2=0x7f3a1406bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:49.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- 172.21.15.133:0/3428255 shutdown_connections 2024-09-16T15:08:49.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- 172.21.15.133:0/3428255 wait complete. 2024-09-16T15:08:49.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 Processor -- start 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- start start 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a140a9ae0 0x7f3a14130af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 0x7f3a1409df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a141365b0 0x7f3a1409e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140a1f10 con 0x7f3a141365b0 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1409eb90 con 0x7f3a140a9ae0 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1409ee50 con 0x7f3a140ab810 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1a914640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a141365b0 0x7f3a1409e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a19912640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 0x7f3a1409df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a19912640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 0x7f3a1409df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55130/0 (socket says 172.21.15.133:55130) 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1a914640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a141365b0 0x7f3a1409e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33348/0 (socket says 172.21.15.133:33348) 2024-09-16T15:08:49.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a19912640 1 -- 172.21.15.133:0/4037183324 learned_addr learned my addr 172.21.15.133:0/4037183324 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1a113640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a140a9ae0 0x7f3a14130af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a19912640 1 -- 172.21.15.133:0/4037183324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a140a9ae0 msgr2=0x7f3a14130af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a19912640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a140a9ae0 0x7f3a14130af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a19912640 1 -- 172.21.15.133:0/4037183324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a141365b0 msgr2=0x7f3a1409e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a19912640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a141365b0 0x7f3a1409e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a19912640 1 -- 172.21.15.133:0/4037183324 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100094d0 con 0x7f3a140ab810 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1a914640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a141365b0 0x7f3a1409e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a19912640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 0x7f3a1409df70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a100ae040 tx=0x7f3a10019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a0affd640 1 -- 172.21.15.133:0/4037183324 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100aa970 con 0x7f3a140ab810 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a0affd640 1 -- 172.21.15.133:0/4037183324 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3a100aaad0 con 0x7f3a140ab810 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a0affd640 1 -- 172.21.15.133:0/4037183324 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100aae80 con 0x7f3a140ab810 2024-09-16T15:08:49.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- 172.21.15.133:0/4037183324 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a14130090 con 0x7f3a140ab810 2024-09-16T15:08:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- 172.21.15.133:0/4037183324 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1412fb10 con 0x7f3a140ab810 2024-09-16T15:08:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.000+0000 7f3a1c377640 1 -- 172.21.15.133:0/4037183324 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a140672a0 con 0x7f3a140ab810 2024-09-16T15:08:49.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.004+0000 7f3a0affd640 1 -- 172.21.15.133:0/4037183324 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 14) v1 ==== 128715+0+0 (secure 0 0 0) 0x7f3a100b2070 con 0x7f3a140ab810 2024-09-16T15:08:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.004+0000 7f3a0affd640 20 mgrc handle_mgr_map mgrmap(e 14) v1 2024-09-16T15:08:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.004+0000 7f3a0affd640 4 mgrc handle_mgr_map Got map version 14 2024-09-16T15:08:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.004+0000 7f3a0affd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.004+0000 7f3a0affd640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.004+0000 7f3a0affd640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f39f8097690 0x7f39f8099b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.004+0000 7f3a0affd640 1 -- 172.21.15.133:0/4037183324 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f3a1015a600 con 0x7f3a140ab810 2024-09-16T15:08:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.004+0000 7f3a1a113640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f39f8097690 0x7f39f8099b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:49.009 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.004+0000 7f3a1a113640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f39f8097690 0x7f39f8099b50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400b810 tx=0x7f3a04008040 comp rx=0 tx=0).ready entity=mgr.4504 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:49.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.008+0000 7f3a0affd640 1 -- 172.21.15.133:0/4037183324 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3a10127cd0 con 0x7f3a140ab810 2024-09-16T15:08:49.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.184+0000 7f3a1c377640 1 -- 172.21.15.133:0/4037183324 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3a141305b0 con 0x7f3a140ab810 2024-09-16T15:08:49.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.488+0000 7f3a0affd640 1 -- 172.21.15.133:0/4037183324 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mgrmap(e 15) v1 ==== 128923+0+0 (secure 0 0 0) 0x7f3a101280a0 con 0x7f3a140ab810 2024-09-16T15:08:49.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.488+0000 7f3a0affd640 20 mgrc handle_mgr_map mgrmap(e 15) v1 2024-09-16T15:08:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.488+0000 7f3a0affd640 4 mgrc handle_mgr_map Got map version 15 2024-09-16T15:08:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.488+0000 7f3a0affd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:49.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.868+0000 7f3a0affd640 1 -- 172.21.15.133:0/4037183324 <== mon.2 v2:172.21.15.133:3301/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3a1013b080 con 0x7f3a140ab810 2024-09-16T15:08:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 10 mgrc shutdown 2024-09-16T15:08:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 -- 172.21.15.133:0/4037183324 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f39f8097690 msgr2=0x7f39f8099b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f39f8097690 0x7f39f8099b50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400b810 tx=0x7f3a04008040 comp rx=0 tx=0).stop 2024-09-16T15:08:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 -- 172.21.15.133:0/4037183324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 msgr2=0x7f3a1409df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 0x7f3a1409df70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a100ae040 tx=0x7f3a10019180 comp rx=0 tx=0).stop 2024-09-16T15:08:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 -- 172.21.15.133:0/4037183324 shutdown_connections 2024-09-16T15:08:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f39f8097690 0x7f39f8099b50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:49.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a141365b0 0x7f3a1409e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:49.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a140ab810 0x7f3a1409df70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:49.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 --2- 172.21.15.133:0/4037183324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a140a9ae0 0x7f3a14130af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:49.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 -- 172.21.15.133:0/4037183324 >> 172.21.15.133:0/4037183324 conn(0x7f3a1406a030 msgr2=0x7f3a140a7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:49.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 -- 172.21.15.133:0/4037183324 shutdown_connections 2024-09-16T15:08:49.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:49.872+0000 7f3a1c377640 1 -- 172.21.15.133:0/4037183324 wait complete. 2024-09-16T15:08:49.897 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:08:50.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.020+0000 7f2f7f8d7640 1 Processor -- start 2024-09-16T15:08:50.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.020+0000 7f2f7f8d7640 1 -- start start 2024-09-16T15:08:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.020+0000 7f2f7f8d7640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f780a6300 0x7f2f780a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.020+0000 7f2f7f8d7640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2f78140830 0x7f2f7814a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.020+0000 7f2f7f8d7640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f780b6320 con 0x7f2f780b5f40 2024-09-16T15:08:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.020+0000 7f2f7f8d7640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f780a8530 con 0x7f2f780a6300 2024-09-16T15:08:50.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.020+0000 7f2f7f8d7640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f78143a20 con 0x7f2f78140830 2024-09-16T15:08:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f7ce72640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f780a6300 0x7f2f780a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f7ce72640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f780a6300 0x7f2f780a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33358/0 (socket says 172.21.15.133:33358) 2024-09-16T15:08:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f7ce72640 1 -- 172.21.15.133:0/2635276910 learned_addr learned my addr 172.21.15.133:0/2635276910 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f7de74640 1 --2- 172.21.15.133:0/2635276910 >> v2:172.21.15.191:3300/0 conn(0x7f2f78140830 0x7f2f7814a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f7ce72640 1 -- 172.21.15.133:0/2635276910 >> v2:172.21.15.191:3300/0 conn(0x7f2f78140830 msgr2=0x7f2f7814a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f7ce72640 1 --2- 172.21.15.133:0/2635276910 >> v2:172.21.15.191:3300/0 conn(0x7f2f78140830 0x7f2f7814a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f7ce72640 1 -- 172.21.15.133:0/2635276910 >> v1:172.21.15.191:6789/0 conn(0x7f2f780b5f40 legacy=0x7f2f780a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:50.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f7ce72640 1 -- 172.21.15.133:0/2635276910 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f78143cc0 con 0x7f2f780a6300 2024-09-16T15:08:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f7ce72640 1 --2- 172.21.15.133:0/2635276910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f780a6300 0x7f2f780a7ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f780a3070 tx=0x7f2f7401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ae6e865afc8963 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f6ffff640 1 -- 172.21.15.133:0/2635276910 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740ab650 con 0x7f2f780a6300 2024-09-16T15:08:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f6ffff640 1 -- 172.21.15.133:0/2635276910 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2f78143810 con 0x7f2f780a6300 2024-09-16T15:08:50.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.024+0000 7f2f6ffff640 1 -- 172.21.15.133:0/2635276910 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740ab9f0 con 0x7f2f780a6300 2024-09-16T15:08:50.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- 172.21.15.133:0/2635276910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f780a6300 msgr2=0x7f2f780a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:50.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 --2- 172.21.15.133:0/2635276910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f780a6300 0x7f2f780a7ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f780a3070 tx=0x7f2f7401c430 comp rx=0 tx=0).stop 2024-09-16T15:08:50.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- 172.21.15.133:0/2635276910 shutdown_connections 2024-09-16T15:08:50.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 --2- 172.21.15.133:0/2635276910 >> v2:172.21.15.191:3300/0 conn(0x7f2f78140830 0x7f2f7814a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:50.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 --2- 172.21.15.133:0/2635276910 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f780a6300 0x7f2f780a7ff0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:50.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- 172.21.15.133:0/2635276910 >> 172.21.15.133:0/2635276910 conn(0x7f2f78065d00 msgr2=0x7f2f78066110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:50.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- 172.21.15.133:0/2635276910 shutdown_connections 2024-09-16T15:08:50.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- 172.21.15.133:0/2635276910 wait complete. 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 Processor -- start 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- start start 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f780a6300 0x7f2f7813ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f780b5f40 0x7f2f78071900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f78140830 0x7f2f78142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7809f2f0 con 0x7f2f78140830 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7813f330 con 0x7f2f780a6300 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7806f0b0 con 0x7f2f780b5f40 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7de74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f78140830 0x7f2f78142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7de74640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f78140830 0x7f2f78142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33360/0 (socket says 172.21.15.133:33360) 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7de74640 1 -- 172.21.15.133:0/1307471816 learned_addr learned my addr 172.21.15.133:0/1307471816 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:50.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7ce72640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f780b5f40 0x7f2f78071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7d673640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f780a6300 0x7f2f7813ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7ce72640 1 -- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f780a6300 msgr2=0x7f2f7813ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7ce72640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f780a6300 0x7f2f7813ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7ce72640 1 -- 172.21.15.133:0/1307471816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f78140830 msgr2=0x7f2f78142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7ce72640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f78140830 0x7f2f78142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7ce72640 1 -- 172.21.15.133:0/1307471816 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f740094d0 con 0x7f2f780b5f40 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7d673640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f780a6300 0x7f2f7813ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7de74640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f78140830 0x7f2f78142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7ce72640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f780b5f40 0x7f2f78071900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f740ac000 tx=0x7f2f7401ac70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f6dffb640 1 -- 172.21.15.133:0/1307471816 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740b4800 con 0x7f2f780b5f40 2024-09-16T15:08:50.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f6dffb640 1 -- 172.21.15.133:0/1307471816 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2f740b4960 con 0x7f2f780b5f40 2024-09-16T15:08:50.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- 172.21.15.133:0/1307471816 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7806f370 con 0x7f2f780b5f40 2024-09-16T15:08:50.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f6dffb640 1 -- 172.21.15.133:0/1307471816 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740b4d30 con 0x7f2f780b5f40 2024-09-16T15:08:50.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.028+0000 7f2f7f8d7640 1 -- 172.21.15.133:0/1307471816 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7809f740 con 0x7f2f780b5f40 2024-09-16T15:08:50.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.032+0000 7f2f637fe640 1 -- 172.21.15.133:0/1307471816 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f50005050 con 0x7f2f780b5f40 2024-09-16T15:08:50.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.032+0000 7f2f6dffb640 1 -- 172.21.15.133:0/1307471816 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 15) v1 ==== 128923+0+0 (secure 0 0 0) 0x7f2f740b3380 con 0x7f2f780b5f40 2024-09-16T15:08:50.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.032+0000 7f2f6dffb640 20 mgrc handle_mgr_map mgrmap(e 15) v1 2024-09-16T15:08:50.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.032+0000 7f2f6dffb640 4 mgrc handle_mgr_map Got map version 15 2024-09-16T15:08:50.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.032+0000 7f2f6dffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:50.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.032+0000 7f2f6dffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:50.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.032+0000 7f2f6dffb640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f2f5c0978d0 0x7f2f5c099d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:50.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.032+0000 7f2f6dffb640 1 -- 172.21.15.133:0/1307471816 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f2f7415a3d0 con 0x7f2f780b5f40 2024-09-16T15:08:50.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.032+0000 7f2f7d673640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f2f5c0978d0 0x7f2f5c099d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:50.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.032+0000 7f2f7d673640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f2f5c0978d0 0x7f2f5c099d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2f700045c0 tx=0x7f2f70009040 comp rx=0 tx=0).ready entity=mgr.4504 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:50.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.036+0000 7f2f6dffb640 1 -- 172.21.15.133:0/1307471816 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2f74127770 con 0x7f2f780b5f40 2024-09-16T15:08:50.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.200+0000 7f2f637fe640 1 -- 172.21.15.133:0/1307471816 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2f50005300 con 0x7f2f780b5f40 2024-09-16T15:08:50.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.884+0000 7f2f6dffb640 1 -- 172.21.15.133:0/1307471816 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2f741296a0 con 0x7f2f780b5f40 2024-09-16T15:08:50.887 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:08:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 10 mgrc shutdown 2024-09-16T15:08:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 -- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f2f5c0978d0 msgr2=0x7f2f5c099d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f2f5c0978d0 0x7f2f5c099d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2f700045c0 tx=0x7f2f70009040 comp rx=0 tx=0).stop 2024-09-16T15:08:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 -- 172.21.15.133:0/1307471816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f780b5f40 msgr2=0x7f2f78071900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:50.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f780b5f40 0x7f2f78071900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f740ac000 tx=0x7f2f7401ac70 comp rx=0 tx=0).stop 2024-09-16T15:08:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 -- 172.21.15.133:0/1307471816 shutdown_connections 2024-09-16T15:08:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f2f5c0978d0 0x7f2f5c099d90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f78140830 0x7f2f78142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f780b5f40 0x7f2f78071900 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 --2- 172.21.15.133:0/1307471816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f780a6300 0x7f2f7813ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 -- 172.21.15.133:0/1307471816 >> 172.21.15.133:0/1307471816 conn(0x7f2f78065d00 msgr2=0x7f2f780b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:50.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 -- 172.21.15.133:0/1307471816 shutdown_connections 2024-09-16T15:08:50.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:50.888+0000 7f2f637fe640 1 -- 172.21.15.133:0/1307471816 wait complete. 2024-09-16T15:08:50.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:08:51.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9acb7a9640 1 Processor -- start 2024-09-16T15:08:51.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9acb7a9640 1 -- start start 2024-09-16T15:08:51.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9acb7a9640 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9ac4145680 0x7f9ac414cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:51.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9acb7a9640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac40a9790 con 0x7f9ac4071f30 2024-09-16T15:08:51.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9acb7a9640 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac4144b20 con 0x7f9ac40a7c00 2024-09-16T15:08:51.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9acb7a9640 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac40a6850 con 0x7f9ac4145680 2024-09-16T15:08:51.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac8d44640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9ac4071f30 0x7f9ac4144f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43526/0 (socket says 172.21.15.133:43526) 2024-09-16T15:08:51.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac8d44640 1 -- 172.21.15.133:0/4056398492 learned_addr learned my addr 172.21.15.133:0/4056398492 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-09-16T15:08:51.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac9d46640 1 --2- 172.21.15.133:0/4056398492 >> v2:172.21.15.191:3300/0 conn(0x7f9ac4145680 0x7f9ac414cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042662887 0 0) 0x7f9ac40a9790 con 0x7f9ac4071f30 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab0003530 con 0x7f9ac4071f30 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2134672180 0 0) 0x7f9ab0003530 con 0x7f9ac4071f30 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 >> v2:172.21.15.191:3300/0 conn(0x7f9ac4145680 msgr2=0x7f9ac414cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 --2- 172.21.15.133:0/4056398492 >> v2:172.21.15.191:3300/0 conn(0x7f9ac4145680 0x7f9ac414cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 >> v1:172.21.15.191:6789/0 conn(0x7f9ac40a7c00 legacy=0x7f9ac40a8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac40a6df0 con 0x7f9ac4071f30 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac9d46640 1 --2- 172.21.15.133:0/4056398492 >> v2:172.21.15.191:3300/0 conn(0x7f9ac4145680 0x7f9ac414cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3957950009 0 0) 0x7f9ac0001680 con 0x7f9ac4071f30 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 >> v1:172.21.15.133:6789/0 conn(0x7f9ac4071f30 legacy=0x7f9ac4144f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:51.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab0003980 0x7f9ab0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:51.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab00047d0 0x7f9ab0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:51.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ab0005190 0x7f9ab000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:51.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40a6850 con 0x7f9ab00047d0 2024-09-16T15:08:51.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab000c0e0 con 0x7f9ab0005190 2024-09-16T15:08:51.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab000c2a0 con 0x7f9ab0003980 2024-09-16T15:08:51.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac8d44640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab00047d0 0x7f9ab0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:51.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac9545640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab0003980 0x7f9ab0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:51.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac9d46640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ab0005190 0x7f9ab000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:51.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac8d44640 1 -- 172.21.15.133:0/4056398492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab0003980 msgr2=0x7f9ab0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac8d44640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab0003980 0x7f9ab0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac8d44640 1 -- 172.21.15.133:0/4056398492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ab0005190 msgr2=0x7f9ab000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:08:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac8d44640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ab0005190 0x7f9ab000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac8d44640 1 -- 172.21.15.133:0/4056398492 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab0003530 con 0x7f9ab00047d0 2024-09-16T15:08:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac9d46640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ab0005190 0x7f9ab000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:08:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.036+0000 7f9ac9d46640 1 -- 172.21.15.133:0/4056398492 reap_dead start 2024-09-16T15:08:51.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9ac8d44640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab00047d0 0x7f9ab0004c50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000c240 tx=0x7f9ac001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62515cdd8e0bdd9f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:51.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0005ae0 con 0x7f9ab00047d0 2024-09-16T15:08:51.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9ac00ab070 con 0x7f9ab00047d0 2024-09-16T15:08:51.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9abbfff640 1 -- 172.21.15.133:0/4056398492 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0007070 con 0x7f9ab00047d0 2024-09-16T15:08:51.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 -- 172.21.15.133:0/4056398492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab00047d0 msgr2=0x7f9ab0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:51.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab00047d0 0x7f9ab0004c50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000c240 tx=0x7f9ac001f3b0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 -- 172.21.15.133:0/4056398492 shutdown_connections 2024-09-16T15:08:51.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 --2- 172.21.15.133:0/4056398492 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ab00047d0 0x7f9ab0004c50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 -- 172.21.15.133:0/4056398492 >> 172.21.15.133:0/4056398492 conn(0x7f9ac406f000 msgr2=0x7f9ac406f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:51.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 -- 172.21.15.133:0/4056398492 shutdown_connections 2024-09-16T15:08:51.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 -- 172.21.15.133:0/4056398492 wait complete. 2024-09-16T15:08:51.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 Processor -- start 2024-09-16T15:08:51.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 -- start start 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab00047d0 0x7f9ac41d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ac4140470 0x7f9ac40b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b17e0 0x7f9ac40b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40b2310 con 0x7f9ac40b17e0 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41da2e0 con 0x7f9ac4140470 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9acb7a9640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41da4f0 con 0x7f9ab00047d0 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9ac9545640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab00047d0 0x7f9ac41d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9ac9545640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab00047d0 0x7f9ac41d9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55164/0 (socket says 172.21.15.133:55164) 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9ac9545640 1 -- 172.21.15.133:0/3171158233 learned_addr learned my addr 172.21.15.133:0/3171158233 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.040+0000 7f9ac9d46640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b17e0 0x7f9ac40b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ac8d44640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ac4140470 0x7f9ac40b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:51.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ac9545640 1 -- 172.21.15.133:0/3171158233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ac4140470 msgr2=0x7f9ac40b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ac9545640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ac4140470 0x7f9ac40b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ac9545640 1 -- 172.21.15.133:0/3171158233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b17e0 msgr2=0x7f9ac40b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ac9545640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b17e0 0x7f9ac40b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ac9545640 1 -- 172.21.15.133:0/3171158233 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac41da7b0 con 0x7f9ab00047d0 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ac9545640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab00047d0 0x7f9ac41d9da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400f820 tx=0x7f9ab400fce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ab9ffb640 1 -- 172.21.15.133:0/3171158233 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4001cf0 con 0x7f9ab00047d0 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ab9ffb640 1 -- 172.21.15.133:0/3171158233 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9ab40052e0 con 0x7f9ab00047d0 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9acb7a9640 1 -- 172.21.15.133:0/3171158233 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac40af220 con 0x7f9ab00047d0 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ab9ffb640 1 -- 172.21.15.133:0/3171158233 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400d970 con 0x7f9ab00047d0 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9acb7a9640 1 -- 172.21.15.133:0/3171158233 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac40b2c70 con 0x7f9ab00047d0 2024-09-16T15:08:51.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9acb7a9640 1 -- 172.21.15.133:0/3171158233 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac40b7910 con 0x7f9ab00047d0 2024-09-16T15:08:51.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ab9ffb640 1 -- 172.21.15.133:0/3171158233 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 15) v1 ==== 128923+0+0 (secure 0 0 0) 0x7f9ab400dad0 con 0x7f9ab00047d0 2024-09-16T15:08:51.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ab9ffb640 20 mgrc handle_mgr_map mgrmap(e 15) v1 2024-09-16T15:08:51.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ab9ffb640 4 mgrc handle_mgr_map Got map version 15 2024-09-16T15:08:51.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ab9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:51.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ab9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:51.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ab9ffb640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f9aa80978d0 0x7f9aa8099d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:08:51.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ab9ffb640 1 -- 172.21.15.133:0/3171158233 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4817+0+0 (secure 0 0 0) 0x7f9ab40c08d0 con 0x7f9ab00047d0 2024-09-16T15:08:51.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.044+0000 7f9ac8d44640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f9aa80978d0 0x7f9aa8099d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:08:51.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.048+0000 7f9ac8d44640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f9aa80978d0 0x7f9aa8099d90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00af000 tx=0x7f9ac00aa000 comp rx=0 tx=0).ready entity=mgr.4504 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:08:51.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.048+0000 7f9ab9ffb640 1 -- 172.21.15.133:0/3171158233 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9ab408de20 con 0x7f9ab00047d0 2024-09-16T15:08:51.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.208+0000 7f9acb7a9640 1 -- 172.21.15.133:0/3171158233 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9ac4144f70 con 0x7f9ab00047d0 2024-09-16T15:08:51.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.492+0000 7f9ab9ffb640 1 -- 172.21.15.133:0/3171158233 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mgrmap(e 16) v1 ==== 128979+0+0 (secure 0 0 0) 0x7f9ab408e1f0 con 0x7f9ab00047d0 2024-09-16T15:08:51.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.492+0000 7f9ab9ffb640 20 mgrc handle_mgr_map mgrmap(e 16) v1 2024-09-16T15:08:51.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.492+0000 7f9ab9ffb640 4 mgrc handle_mgr_map Got map version 16 2024-09-16T15:08:51.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.492+0000 7f9ab9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] 2024-09-16T15:08:51.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.888+0000 7f9ab9ffb640 1 -- 172.21.15.133:0/3171158233 <== mon.2 v2:172.21.15.133:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9ab4085d80 con 0x7f9ab00047d0 2024-09-16T15:08:51.892 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 10 mgrc shutdown 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 -- 172.21.15.133:0/3171158233 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f9aa80978d0 msgr2=0x7f9aa8099d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f9aa80978d0 0x7f9aa8099d90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00af000 tx=0x7f9ac00aa000 comp rx=0 tx=0).stop 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 -- 172.21.15.133:0/3171158233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab00047d0 msgr2=0x7f9ac41d9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab00047d0 0x7f9ac41d9da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400f820 tx=0x7f9ab400fce0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 -- 172.21.15.133:0/3171158233 shutdown_connections 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.191:6824/4181899780,v1:172.21.15.191:6825/4181899780] conn(0x7f9aa80978d0 0x7f9aa8099d90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ac40b17e0 0x7f9ac40b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ac4140470 0x7f9ac40b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 --2- 172.21.15.133:0/3171158233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9ab00047d0 0x7f9ac41d9da0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:08:51.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 -- 172.21.15.133:0/3171158233 >> 172.21.15.133:0/3171158233 conn(0x7f9ac406f000 msgr2=0x7f9ac4142c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:08:51.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 -- 172.21.15.133:0/3171158233 shutdown_connections 2024-09-16T15:08:51.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-16T15:08:51.892+0000 7f9acb7a9640 1 -- 172.21.15.133:0/3171158233 wait complete. 2024-09-16T15:08:51.920 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T15:08:51.920 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:08:51.921 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:51.930 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:08:51.930 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:08:51.930 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:51.983 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:08:51.984 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:08:51.984 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:51.996 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:08:51.996 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T15:08:51.996 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T15:08:51.996 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:55.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-16T15:08:58.098 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T15:08:58.098 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T15:08:58.098 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:58.152 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T15:08:58.152 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T15:08:58.153 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:58.241 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T15:08:58.242 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T15:08:58.242 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:58.307 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T15:08:58.307 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T15:08:58.308 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T15:08:58.308 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:58.338 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T15:08:58.338 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T15:08:58.338 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:58.367 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T15:08:58.368 DEBUG:tasks.ceph.mgr.z:waiting for process to exit 2024-09-16T15:08:58.368 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:58.437 INFO:tasks.ceph.mgr.z:Stopped 2024-09-16T15:08:58.437 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T15:08:58.438 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T15:08:58.438 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:58.452 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T15:08:58.452 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T15:08:58.452 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:58.512 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T15:08:58.513 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T15:08:58.513 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:08:58.535 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T15:08:58.535 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T15:08:58.535 DEBUG:teuthology.orchestra.run.smithi133:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v 'replacing it with standby' | egrep -v 'No standby daemons available' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-16T15:08:58.567 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-16T15:08:58.567 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T15:08:58.921 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-16T15:08:58.921 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T15:08:59.374 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-16T15:08:59.374 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T15:08:59.553 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi191.front.sepia.ceph.com 2024-09-16T15:08:59.554 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T15:08:59.710 INFO:tasks.ceph:Archiving mon data... 2024-09-16T15:08:59.711 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795/data/mon.a.tgz 2024-09-16T15:08:59.712 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-16T15:08:59.718 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.I30Su5AeTZ 2024-09-16T15:08:59.718 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.I30Su5AeTZ 2024-09-16T15:08:59.920 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.I30Su5AeTZ 2024-09-16T15:08:59.983 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.I30Su5AeTZ is 912KB 2024-09-16T15:09:00.048 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.I30Su5AeTZ 2024-09-16T15:09:00.054 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795/data/mon.c.tgz 2024-09-16T15:09:00.054 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-16T15:09:00.106 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.jxdUhRySU8 2024-09-16T15:09:00.106 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.jxdUhRySU8 2024-09-16T15:09:00.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-16T15:09:00.383 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.jxdUhRySU8 2024-09-16T15:09:00.446 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.jxdUhRySU8 is 1MB 2024-09-16T15:09:00.516 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.jxdUhRySU8 2024-09-16T15:09:00.522 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795/data/mon.b.tgz 2024-09-16T15:09:00.522 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-09-16T15:09:00.528 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.qDi9OBQxiQ 2024-09-16T15:09:00.528 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.qDi9OBQxiQ 2024-09-16T15:09:00.803 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.qDi9OBQxiQ 2024-09-16T15:09:00.865 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.qDi9OBQxiQ is 1MB 2024-09-16T15:09:00.935 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.qDi9OBQxiQ 2024-09-16T15:09:00.941 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T15:09:00.941 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T15:09:00.944 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T15:09:00.997 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-16T15:09:00.998 DEBUG:teuthology.orchestra.run.smithi133:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T15:09:01.000 DEBUG:teuthology.orchestra.run.smithi191:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T15:09:01.017 INFO:teuthology.orchestra.run.smithi133.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T15:09:01.072 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T15:09:01.076 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T15:09:01.078 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795/remote/smithi133/crash 2024-09-16T15:09:01.079 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T15:09:01.090 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795/remote/smithi191/crash 2024-09-16T15:09:01.091 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T15:09:01.131 INFO:tasks.ceph:Shutting down logrotate 2024-09-16T15:09:01.131 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:09:01.135 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:09:01.216 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-16T15:09:01.228 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-16T15:09:01.269 INFO:tasks.ceph:Compressing logs... 2024-09-16T15:09:01.269 DEBUG:teuthology.orchestra.run.smithi133:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:09:01.271 DEBUG:teuthology.orchestra.run.smithi191:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:09:03.992 INFO:tasks.ceph:Archiving logs... 2024-09-16T15:09:03.993 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795/remote/smithi133/log 2024-09-16T15:09:03.994 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T15:09:04.202 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795/remote/smithi191/log 2024-09-16T15:09:04.227 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T15:09:04.484 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T15:09:04.493 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T15:09:04.494 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T15:09:04.496 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T15:09:04.553 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T15:09:04.561 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T15:09:04.561 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:09:04.564 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:09:04.575 INFO:teuthology.orchestra.run.smithi133.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T15:09:04.575 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================== 2024-09-16T15:09:04.576 INFO:teuthology.orchestra.run.smithi133.stdout:*hv01.front.sepi 67.205.162.81 3 u 54 64 17 0.104 -2.586 1.360 2024-09-16T15:09:04.576 INFO:teuthology.orchestra.run.smithi133.stdout: hv02.front.sepi 63.231.80.2 3 u 49 64 17 0.057 -1.826 1.233 2024-09-16T15:09:04.576 INFO:teuthology.orchestra.run.smithi133.stdout: hv03.front.sepi 74.6.168.72 3 u 49 64 17 0.108 -2.407 1.206 2024-09-16T15:09:04.576 INFO:teuthology.orchestra.run.smithi133.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T15:09:04.577 INFO:teuthology.orchestra.run.smithi191.stdout: remote refid st t when poll reach delay offset jitter 2024-09-16T15:09:04.577 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================== 2024-09-16T15:09:04.577 INFO:teuthology.orchestra.run.smithi191.stdout: hv01.front.sepi 67.205.162.81 3 u 52 64 17 0.118 -1.652 0.955 2024-09-16T15:09:04.577 INFO:teuthology.orchestra.run.smithi191.stdout: hv02.front.sepi 63.231.80.2 3 u 52 64 17 0.102 -1.098 1.211 2024-09-16T15:09:04.577 INFO:teuthology.orchestra.run.smithi191.stdout:*hv03.front.sepi 74.6.168.72 3 u 52 64 17 0.121 -1.700 0.947 2024-09-16T15:09:04.577 INFO:teuthology.orchestra.run.smithi191.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-16T15:09:04.578 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T15:09:04.586 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T15:09:04.586 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T15:09:04.595 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T15:09:04.603 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T15:09:04.611 INFO:teuthology.task.internal:Duration was 711.956091 seconds 2024-09-16T15:09:04.611 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T15:09:04.620 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T15:09:04.620 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T15:09:04.622 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T15:09:04.660 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T15:09:04.660 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-09-16T15:09:04.660 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T15:09:04.710 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-09-16T15:09:04.710 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T15:09:04.720 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T15:09:04.720 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:09:04.755 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:09:04.786 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T15:09:04.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T15:09:04.823 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T15:09:04.886 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T15:09:04.902 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T15:09:04.902 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T15:09:04.913 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T15:09:04.927 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T15:09:04.942 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:09:04.956 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:09:04.962 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-09-16T15:09:04.979 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-09-16T15:09:05.095 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:09:05.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:09:05.099 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:09:05.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:09:05.141 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T15:09:05.151 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T15:09:05.152 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795/remote/smithi133 2024-09-16T15:09:05.152 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T15:09:05.174 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906795/remote/smithi191 2024-09-16T15:09:05.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T15:09:05.208 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T15:09:05.209 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T15:09:05.216 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T15:09:05.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T15:09:05.260 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T15:09:05.260 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T15:09:05.269 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T15:09:05.269 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T15:09:05.272 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T15:09:05.275 INFO:teuthology.orchestra.run.smithi133.stdout: 658623 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 15:09 /home/ubuntu/cephtest 2024-09-16T15:09:05.297 INFO:teuthology.orchestra.run.smithi191.stdout: 658639 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 15:09 /home/ubuntu/cephtest 2024-09-16T15:09:05.299 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T15:09:05.312 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T15:09:05.332 INFO:teuthology.run:Summary data: description: rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/workunits} duration: 711.9560906887054 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T15:09:05.333 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T15:09:05.415 INFO:teuthology.run:pass